In February 2009, I fell from three stories high and fractured three vetrebrae along with other, more minor injuries. In the hospital, I was underwhelmed by the standard of care – why weren’t practitioners leveraging existing technology? Fast forward 5 years later to the last year of my B.Eng. Physics in 2014 when I formed this team…
Actuation Technology for the External Assistance of Muscles (ATEAM)
Movement is our oldest, most primal form of communication. Body language forms most of human interaction and is the basis age-old activities like dancing, sports, martial arts, and performing music. Yet, although we humans have built vast libraries to store knowledge in the form of audio, video, literature, and even 3D models, our rich history of human kinesthetics is being lost. We pass on this knowledge through indirect ways like word-of-mouth or video tutorials. We watch videos of Maradonna, Bruce Lee, and Michael Jackson to learn as much as we can of their technique by osmosis. So, the ATEAM was formed to develop a method of storing human kinetic information that is direct, portable and accessible. We used a combination of two noninvasive technologies – motion capture (MoCap) to detect motion and electromyography (EMG) to measure the muscle activity during that motion. The system would be integrated into clothes so that it could analyze the wearer’s kinesthetic information on the go. From there, it could also possibly analyze the wearer’s neuromuscular structure and diagnose potential problems. The data would be recorded, visualized, and stored in real-time on a smartphone using our custom app.
So how does this relate to my injury?
The human body is capable of many feats of physical prowess from triathlons to intricate dance choreographies, but as I found out, our bodies are also vulnerable to injuries, disorders and aging, that can damage our neuromuscular system and limit movement by affecting our strength, dexterity, and endurance. Although there is currently no cure for Parkinsons, aging, ALS, etc., the consequences could still be overcome with the right tools. Unfortunately, there is no technology to help weakened limbs because our muscles are so efficient for their size that robotics cannot yet replicate them. On one hand, prosthetics can replace (or even supersede) missing limbs. On the other hand, exoskeleton-based suits for military purposes can boost speed and strength but are too clunky for day-to-day use. Their counterparts (e.g., Axosuit, HAL) are still costly, mechanical and bulky. The efficiency of muscles can be attributed to: their simple functionality, fibrous structure, and solid state movement. All muscle fibres perform only one function – contraction, which allows them to specialize. They can work together to produce an overall twisting or bending motion at a joint. This networked structure also ensures that the muscle itself does not lose functionality when a few fibres tear. Muscle fibers are also solid-state (no moving parts) so that movement is produced by shape-shifting and no energy is lost to friction, unnecessary work, or energy conversions. This is why we designed an artificial muscle system that mimics muscles. Our polymer-based fibres employed a smart material called electro-active polymers. Our rope-like structures act as compliant capacitors that require a high voltage and low current to compress when a voltage is applied. The helical dielectric elastomer (HDEA) fibres can then be ‘woven’ into clothes to enhance a wearer’s movement.
An attractive application of this technology is in the potential to build customizable networks of fibres that can operate in coordinated patterns (similar to groups of muscle fibers) to produce more complex overall movements. These networks could be applied to all forms of free-space robotics such as positioning, manufacturing, surgery and physical therapy. After our Capstone project ended, I moved to a MEng at McGill University. Alongside my MEng, I tried to continue developing this technology via the entrepreneurial route.
Polymer Actuation Technology for Treatment, Empowerment, Rehabilitation, and New Solutions (PATTERNS)
Towards the end of 2015, I entered a start-up pitch competition with 3 M.B.A. students – Ana, Margarita, and Kristov. We decided to use this technology to help people undergoing physiotherapy. Although our pitch didn’t win, it was an eye-opening learning opportunity. So, Kristov and I decided to keep going. We recruited Nahid, a Ph.D. student in rehabilitation science, to complement the expertise of our team. Our start-up, PATTERNS, began forming collaborations in engineering, physiotherapy, and business. We spoke to practitioners and clinicians, developed strong ties with the Jewish Rehabilitation Hospital, supervised two B.Eng. Capstone teams, and became a business case study for an M.B.A. project with Natasha, James, Nico, and Kristov. It was an eye-opening foray into entrepreneurship, which peaked with us winning an Honourable Mention in the Health Sciences track of McGill’s Dobson Cup.
PATTERNS’ flagship product was designed to assist walking. People who suffered a stroke would often also contract Foot Drop. This meant that they lost control of their ankle, causing their foot to go limp and impeding their ability to walk properly. Using solid-state mechanics, our solution offered them a way to maintain a regular gait while walking. Here’s how it worked:
Unfortunately, soon after the Dobson Cup, Kristov met with an accident and PATTERNS dissolved. But I wasn’t ready to give up. I figured that if my own Capstone team could achieve so much, then why not try to continue down that path? So, with the go-ahead from my supervisor, Prof. Dave Plant, I developed a program in which we would co-supervise B.Eng. students for their final year Capstone project. In this arrangement, the students got hands-on experience in the lab, exposure to state-of-the-art equipment, and free training on emerging technologies. In doing so, they were contributing to this venture. Over 5 years, we supervised 100+ students of which Galib, Nathan, Adam, Santi, and James decided to continue with us for an MEng, of which Santi and James have stayed even longer for a Ph.D. Santi did his Capstone on the HDEA networks described above. For his MEng and PhD, he has moved onto a different silicon-based smart material – photonic crystal circuits, which I explain in chapter 2 and 3 of my PhD. James began with the muscle sensors that I described in my TEDx talk shown above. Towards the end of James’ Capstone, we were approached by Neptec (now MDA) who wanted to use our results for…
Fun fact: what’s one of the richest demographics of people who don’t have access to healthcare? Astronauts and helicopter pilots!
To secure our partnership with MDA for space-health, I spent the next 14 months writing 2 proposals for federal reseach funding worth $1M. These grants enabled us to build a start-of-the-art research laboratory for human trials as well as hire James, Galib, Nathan, Angus, and myself as graduate students researchers on the project. So, for James’ MEng and PhD, he specialized in developing a wearable vital signs monitor, which was the next step in the evolution of this venture.
Non-invasive Physiological Activity Monitoring System (NiPAMS)
The four vital signs – heart rate, respiration rate, blood pressure, and body temperature – are the basis of any health evaluation. However, measuring each vital sign requires a different, ‘gold standard’ device (e.g., Holter monitor, spirometer, sphygmomanometer). Each of these devices are either cumbersome or limited to single-shot measurements. Alternatively, the wearable devices (e.g., smart watch, strain gauge) are either inaccurate, unreliable, or are limited to peripheral measurements. So, we designed a wearable sensor which can provide continuous measurements of all four vital signs from a single, central point of contact.
The sensor is placed on the xiphoid process of the sternum. It contains a thermistor, which detects the temperature at the point of contact, i.e., body temperature, and is therefore a nonissue. It also contains an accelerometer and gyroscope, which detect the vibrational motion of the chest in six degrees of freedom. We have already developed motion signal processing algorithms to extract measurements of cardiac [1-6], respiratory [7-11], and hemodynamic [12-14] activity at the heart and lungs. Additionally, we have patented the methods to extract heart rate and respiration rate [1] as well as blood pressure [14]. However, the development of a physically valid blood pressure measurement requires data which we cannot yet obtain.
Our research revealed how vibrational cardiography (VCG) [3] can be used to obtain all four vital signs at the source in real‑time. It demonstrates the potential of VCG for medical-standard health monitoring in real-world scenarios such as telemedicine, providing immediate information to first‑responders, and daily monitoring for the prediction and prevention of cardiorespiratory diseases (which are the leading causes of death in North America). Preliminary simulations also allude to the possibility of using this data to define a digital twin of the vital organs for a given subject, which would enable deeper analysis and diagnostics. It is therefore a building block toward remote, autonomous healthcare solutions which provide accessibility to patients who would otherwise be restricted by time, money, distance, or the availability of resources or expertise.
The sensor captures the mechanical output of the heart, which provides unique insights into cardiovascular dynamics including the pre-ejection period [2], left-ventricular ejection time [4], and the movement of the heart within the chest [5]. This type of information cannot be obtained by ECG or ICG which only capture the electrical input to the heart.
The sensor captures the complex movement of the lungs, which can distinguish between the expansion of the intercostal muscles or the diaphragm [11]. This distinction cannot be obtained by a spirometer or a strain gauge which only capture, respectively, the exhaled air or the expansion of the point of contact.
The sensor captures the movement of the valves including their timing, amplitude, and duration [4, 5]. This provides the basis for deriving hemodynamic measurements directly at the heart and ascending aorta (which cannot be obtained by an upper arm or finger cuff), and non-invasively (which cannot be obtained by an intra-aortic catheter) [13].
Our full prototype costs only $100 with ECG [3] and could be connected to a smartphone via BLE [1]. Such an inexpensive system allows for the mass market to engage in continuous monitoring over extended periods. This data would enable large scale, longitudinal studies of cardiorespiratory activity. The resulting insights would provide the basis for the detection, prediction, and prevention of anything from fatal diseases to minor illnesses concerning the heart and lungs.
Finally, the hardware roadmap offers minimal risk. Since a wireless motion sensor holds its own market value, it can be developed and sold as a base product. Its utility can then be expanded (by the development of its associated signal processing algorithms) by simply updating the smartphone app without needing to switch out or recall the sensor itself. For example, current solutions can also extract measurements from the motion of the smartphone itself if users place their phone on their chest while lying down (which is less reliable, inaccurate, and cannot be used for continuous monitoring).
Here is a talk I did at the IEEE Engineering and Medicine in Biology conference in 2020, which explains the system:
As a final note, allow me to harp on the potential of this technology even further. The largest contributing factor to global mortality rates are curable diseases, mainly due to the inaccessibility of healthcare. Our solution, ‘prevention through prediction’, proposes that health issues can be diagnosed well in advance if real-time physiological readings can be cross-referenced with medical records. This depends on our ability to monitor, analyze, and interpret physiological functions in real-time, and then: (a) calibrate the readings to the subject’s personal physiology, (b) provide feedback to improve accuracy, and (c) refine health trends and models. Through this venture, we hope to overcome a key limitation in current medicine by providing a means for continuous, on-the-go, automated health monitoring. Such a system could eliminate the need for annual check-ups (and thereby reduce the onus on medical professionals), communicate health information to loved ones whenever necessary, provide telemedical support in the case of an emergency, and even predict health issues before they happen.
Note: I’m always open to collaborating! If you would like to try, use, or further develop any of the aforementioned technologies, contact me 🙂
References
[1] Y. D’Mello, P. Roche, and M. Lortie, “Method and Apparatus for Deriving Biometric Information using Multiple-Axis Seismocardiography,” Patent CA2018051006 Patent Appl. CA2018051006, 2018.
[2] Y. D’Mello et al., “Autocorrelated Differential Algorithm for Real-Time Seismocardiography Analysis,” IEEE Sensors Journal, vol. 19, no. 13, pp. 5127-5140, 2019, doi: 10.1109/JSEN.2019.2903449.
[3] Y. D’Mello et al., “Real-Time Cardiac Beat Detection and Heart Rate Monitoring from Combined Seismocardiography and Gyrocardiography,” Sensors, vol. 19, no. 16, p. 3472, 2019. [Online]. Available: https://www.mdpi.com/1424-8220/19/16/3472.
[4] Y. D’Mello et al., “Identification of the Vibrations Corresponding with Heart Sounds using Vibrational Cardiography,” in 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 20-24 July 2020 2020, pp. 17-20, doi: 10.1109/EMBC44109.2020.9175323.
[5] S. Hakim, “Effect of Chest Morphology on Vibrational Cardiography Waveforms using Multi-Sensor Analysis,” Master of Science, Department of Electrical & Computer Engineering McGill University, Montreal, Canada, 2021.
[6] E. Aboulezz et al., “Analyzing Heart Rate Estimation from Vibrational Cardiography Caused with Different Orientations,” in 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2020.
[7] J. Skoric et al., “Relationship of the Respiration Waveform to a Chest Worn Inertial Sensor,” in 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2020.
[8] J. Skoric, Y. D’Mello, M. Lortie, S. Gagnon, and D. V. Plant, “Effect of Static Respiratory Volume on the Waveform of Cardiac-induced Sternal Vibrations,” in 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 23-27 July 2019, pp. 4917-4921, doi: 10.1109/EMBC.2019.8857505.
[9] J. Skoric, “Respiratory Modulation of Sternal Accelerations,” Master of Science, Department of Electrical & Computer Engineering McGill University, Montreal, Canada, 2020.
[10] N. Clairmonte et al., “Neural Network-based Classification of Static Lung Volume States using Vibrational Cardiography,” in 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2020.
[11] J. Skoric et al., “Respiratory Modulation of Sternal Motion in the Context of Seismocardiography,” IEEE Sensors Journal, vol. 22, no. 13, pp. 13055-13066, 2022, doi: 10.1109/JSEN.2022.3173205.
[12] M. Schiavone, “A mechanical model of cardiovascular system for blood pressure estimation,” Master of Mechanical Engineering, Politecnico di Torino, Torino, Italy, 2021.
[13] J. Skoric et al., “Cuff-less Estimation of Blood Pressure from Vibrational Cardiography Using a Convolutional Neural Network,” in 2022 Computing in Cardiology (CinC), 4-7 Sept. 2022 2022, vol. 498, pp. 1-4, doi: 10.22489/CinC.2022.110.
[14] Y. D’Mello, J. Skoric, M. A. Lortie, and D. V. Plant, “Systems, Methods, And Devices For Non-Invasive And Continuous Hemodynamic Measurement,” Canada Patent Appl. 63/033,103, 2021.