The most serious physical disability in children is caused by cerebral palsy (CP), a frequent mobility disease in children. Early diagnosis is essential for an early intervention and it helps in potential recovery of infants at high risk. Diagnosis of cerebral palsy is crucial at an early stage since it allows monitoring and therapy sooner. Children with cerebral palsy are prone to high error and may cause underestimation in the values of Hypothalamic–Pituitary–adrenal that is often detected in children with limitations in mobility. Accelerometer-based motion sensors have been acknowledged as the standard for accurately measuring PA in children and adolescents. GAIT aims to map these readings and create a 3-Dimensional model to map coordinates and perform analysis, however finding the severity and the type of Cerebral Palsy is a task due to a lack of classification models. The paper aims to deploy a classification model to predict the presence, intensity and severity of the condition in infants / adults by using the coordinate dataset provided by the GAIT Lab datasets available within the institute’s GAIT Lab. Alongside this, also focusing on deploying an infant cerebral palsy prediction model that can predict the condition in early stages and be used for treatment.
A person’s balance, mobility and posture are widely affected by a number of diseases together referred to as cerebral palsy (CP). It is the most common kind of motor disability in kids. “Cerebral” refers to matters pertaining to the brain, while “Palsy” is used with respect to muscle weakness or dysfunction. A human with Cerebral Palsy has a brain injury or faulty brain development that affects their ability to regulate their muscles.
Symptoms of a human may vary based on the anatomy. In severe cases of CP, a person who would face the disability of walking might need special walking assistance, in which case they might need lifelong care. While CP at mild levels would cause the person to walk a little awkwardly but special help might not be needed. There could be numerous fluctuations in the disease symptoms but, CP does not get worse over time.
Cerebral Palsy is widely known to affect the movement and posture of the human body while the additional effects may also include suffering through associated disorders such as seizures, speech issues, vision hearing, intellectual disabilities, some may be much more severe than the previously mentioned ones, such as scoliosis or contractures, etc. It is estimated that around 3 out to 10 children develop some form of Cerebral Palsy which is widely affected by the gestational age during the birth, hence the early diagnosis acts as a key to the problem.
With the creation of several innovative tools, including early detection of general movement assessments (GMA), research on cerebral palsy is becoming active. The definition of cerebral palsy (CP) is frequently given as” a collection of persistent issues with mobility development and due to non-progressive abnormalities, that arose in the growing body, posture that limits activity in the neonatal or fetal brain” [9]. These syndromes are common sensory, cognitive, communicative, and psychological problems. Epilepsy and secondary syphilis can be severe. There can be musculo-skeletal anomalies [9]. Early detection and cerebral palsy rehabilitation training is especially important for kids.
To find the impact of cerebral palsy in infants and to provide with early detection that can help in the early medication and treatment would be useful, one of the methods deployed is the MINI-RGB dataset, through which several tests can be conducted for help in prediction.
Gait Analysis: There is potential for using Biometric Gait Analysis as a system alternative for biometric authentication. A number of a variety of techniques exist for recognizing human gait. These methods use machine vision strategies that cameras with the ability to record gait metrics a range of sensor methods and angles, which while they are connected to the device, they can gather data about gait as a person [12]. Being a research- based initiative, our major emphasis is on accuracy. When the appropriate level of precision would be achieved, the phase to enable real-time utilization of the system would be worked upon. For this project, which is based on machine vision, the aim is to achieve the precision of a sensor-based gait identification system apparatus [4, 11].
Initially, in the novel months of 1940 was when the research was initiated on Cerebral Palsy by a man named Arnold E. Joyal where he stated that Cerebral Palsy is one of the leading reasons for poor and slow-paced growth in infants and children. Due to which it led to subsequent research on this topic such as how cognitive functioning was affected in children whose age was around a mean of nine also having Cerebral Palsy in 1958 by Paul E. Polani. D. F. Parker etal. Measured and calculated the relations found between the motor functions and physiological functions or the fitness in children who are affected by Cerebral Palsy, in the year 1993. Based on the recent papers, researchers have seemed to have come a long way in identifying it in an early stage. In [9] “Deep Learning of fMRI” how fMRI helps us in the study of neural connections in the brain and how it reorganizes itself after preliminary brain injuries and finding out neuroplasticity. By Diagnosing the early stages of infants suffering from Cerebral Palsy, preferably in a span of six to twelve weeks old by looking at fMRI images of these infants and comparing them with help of responses from an oculomotor. Furthermore, classifying into types namely ataxic, dyskinetic, spastic and Mixed cerebral palsy using DNN. New architecture of CNN that is built upon AlexNet that includes five layers of convolution and layers of pooling that are together stacked with the objective of getting a high-level accuracy in classification fMRI brain images of infants with Cerebral Palsy.
The Huge Corpus of studies prepare a base for more future work to be developed [4]. Developing clinical practice guidelines (CPG) and the use of Instrumented Gait Analysis (IGA) for the clinical management of infants/children with cerebral palsy associated with gait disability. Out of the total 909 studies met with inclusion standard and were allocated to six study Criteria’s. IGA was employed as a treatment outcome measure in 49% of research, with the use of subgroup analyses or modelling accounting for 33% of studies on reliability and validity, 2% of studies on clinical decision-making, 2% of studies on therapies that depend on IGA, and 1% of studies on cost effectiveness.
For this categorization challenge, [11] they suggested a fresh deep learning system. They also suggested a framework for visualizing the bodily sections that contribute most to classification decisions. A crucial step toward automation is the incorporation of a visualization framework since it makes the conclusions reached by the machine learning framework understandable. Developed a new and difficult dataset made up of recordings of preterm infants thought to be at a higher risk of developing cerebral palsy to facilitate evaluation of the suggested framework. According to experimental findings, the suggested method outperforms the baseline methods in terms of accuracy and robustness while also offering an understandable depiction of the classification-affecting elements.
Propose a channel attention [6] module for deep learning models to highlight key features (such as body joints) the model identifies as significant, thereby indicating why specific diagnostic results are found. This is done to predict cerebral palsy from infants’ body movements. comprehend the circumstances. Finding a model, algorithm, logic, or analysis to solve the current problem. To forecast the future, deployment of some machine learning algorithms would be done. Linear Regression Model and k-nearest neighbor (KNN) algorithms are examples of supervised machine learning. Different kinds of data sets and algorithms exist.
Gait stability testing is required for kids with CP. [2] The Kinect has the potential to be used as a more affordable gait stability assessment tool than conventional motion capture methods, ensuring continuous and frequent gait monitoring. These findings strongly imply that the Kinect V2 could be used as a low-cost, portable gait assessment tool to screen and track the gait progression of children with cerebral palsy, particularly in terms of the spatiotemporal component. More sensors and reliable algorithms should be used in future studies to improve the accuracy of skeleton tracking, gait event detection, and MOS computation.
To analyze movements of body joints for prediction and early detection.
To create and evaluate machine learning models to predict and analyze the physical disability in ambulant children and adolescents with Cerebral Palsy.
To test the viability and sustainability of machine learning methods to predict the physical disability in children with Cerebral Palsy.
For obtaining the coordinates from the datasets, usage a multi-person, pose detection software, openpose was made, which identifies 18 keypoint in from the RGBD images of the infants and converts them to 2D coordinate features for each joint and limbic connections. Due to the high accuracy and low sensitivity, it is used to extract the coordinates in the form of (x,y) from the video sequences.
The model proposed consists of two major functional parts, each performing a series of operations before the usage of next:
a. Fast fourier transform:
Chirp-z fourier transform is used to convert the results of the OpenPose to frequencies depending on the intensity of the movement. The algorithm is used to compute DFTs of prime length in O(NlogN) time complexity. In our model it performs the transformation of the 2D coordinates into the original joints position features \[\label{GrindEQ__1_} X_K\mathrm{=}\sum\limits_{n\mathrm{=0}}{N\mathrm{-}\mathrm{1}x_ne^{\frac{\mathrm{-}i\mathrm{2}\pi kn}{N}},k\mathrm{=0,\dots ,}N\mathrm{-}\mathrm{1}} . \tag{1}\]
b. Binning strategy:
The above-mentioned binning strategy is used to classify the frequencies based on the function based on the widths of the bands that is received by the coefficients after the application of the Fourier transform while emphasizing on the low-frequency band of values.
\[\label{GrindEQ__2_} b_n=\left\{ \begin{array}{c} \begin{array}{cc} \mathrm{Round}\left(x_0.y_n\right), & x_0.y^n<3 ,\\ & \end{array} \\ \mathrm{Ceiling}\left(x_0.y_n\right),\ {\ \ \ \ \ x}_0.y^n\ge 3,\end{array} \right. \tag{2}\] here, x\({}_{n}\) denotes the width of n\({}_{th}\) bin, x\({}_{0}\) = 1, and y called the controllable parameter.
The literature survey focuses on the usage of traditional convolutional neural networks for the above study, the discrete networks have proved to not have been suitable enough for this proposition due to their incapability of handling graphs to display a common human skeletal graph. A Graph Convolutional neural network is the first layer seen in Figure 1 when used aligns with the infant’s skeletal structure which can be used to interpret movements. The graph G = (V, E) is used to perform the previously mentioned task. The algorithm majorly focuses on the usage. In the graph, \(\mathrm{\{}\)V = v\({}_{bxi}\) x = 1, …, X; i = 1, …, N \(\mathrm{\}}\) signifies the frequencies of the coordinates of the joints retired via the fourier transform, where v\({}_{xi}\) represents the x-th frequency bin of the i\({}^{th}\) joint.
The edges include:
• The intra skeleton connection at each range of frequencies, \(\mathrm{\{}\)v\({}_{bi}\) v\({}_{xi}\)(i, j) \(\mathrm{\in }\) K\(\mathrm{\}}\) here K is used to demonstrate the anatomical structure of the human joint.
• Inter-frequency edges to connect frequency bins of a joint in low-high order \(\mathrm{\{}\)v\({}_{xi}\) v\({}_{x}\)\({}_{+1}\)\(\mathrm{\}}\).
The propagation formula for transfer of information between layers can be demonstrated by: \[\label{GrindEQ__3_} I^{(l+1)}=\sigma ({\tilde{\mathop{E}}}^{-\frac{1}{2}}\tilde{\mathop{B}}{\mathop{F}^{\sim }}^{-\frac{1}{2}}I^{(l)}X^{(m)}) , \tag{3}\] where \(\tilde{\mathop{B}}\)\({}^{\ }\)= B + m\({}_{L}\) denotes the adjacency matrix of the created undirected graph. I\({}_{L}\) is a L dimensioned identity matrix. \(\tilde{\mathop{E}}=\sum\limits_j{{\tilde{\mathop{B}}}_{ij}}\) and X\({}_{(}\)\({}_{l}\)\({}_{)}\) denotes a learnable weight matrix specific to the layer of the network. The nonlinear activation function \(\sigma\)(·) is set as ReLU in our network as seen in Figure 1. Summation operation on all the frequencies is obtained via this particular methodology h\({}_{1,i,}\) h\({}_{2,i,}\) …, h\({}_{B}\), with attentions \(\alpha_{b,i}\) by \[\label{GrindEQ__4_} v_k\mathrm{=}\sum^B_{b\mathrm{=1}}{{\alpha }_{b,i}h_{b,i}} , \tag{4}\] in the above equation the frequency attention weight \(\alpha\)\({}_{b,i}\) is defined as: \[\begin{aligned} \label{GrindEQ__5_} {\alpha }_{b,i}\mathrm{=}&\frac{{\mathrm{exp} \mathrm{(}{\sigma }^{\mathrm{'}}_n\mathrm{(}w^T_{\alpha },z_{b,i}\mathrm{))}\ }}{\sum\limits_b{\mathrm{exp}\mathrm{}\mathrm{(}{\sigma }^{\mathrm{'}}_n\mathrm{(}w^T_{\alpha },z_{b,i}\mathrm{))}}}, \notag\\ z_{b,i}=&\mathrm{tanh}\mathrm{}W_zh_{b,i}, \end{aligned}\ \tag{5}\] here the \(\sigma\)\(\mathrm{\prime}\)\({}_{n}\) denotes an activation function (adjustable) as:
\[\label{GrindEQ__6_} {\sigma }'_n=\left\{ \begin{array}{cc} 1+{(\frac{w_{\alpha }}{\|w_{\alpha }\|})}^T(\frac{z_{\alpha }}{\|z_{\alpha }\|}) & n=1, \\ w^T_{\alpha }z_{\alpha } & n=2 ,\end{array}\right. \tag{6}\] where w\({}_{\alpha }\) and W\({}_{z}\) display the learnable parameters.
Adam Optimization is an algorithm used to optimize neural networks (ANN/RNN/CNN). It can handle large datasets and their parameters well. It is also efficient in handling noise and irregularities. Modifications of model parameters in a systematic way i.e tuning is very important in a model. Adam requires very little tuning. This method is a combination of concepts like momentum and learning rate decay.
It maintains adaptive learning rates separate from each parameter and keeps a single learning rate for all weight updates that remain constant during the training. Adam simply estimates the first two moments of the gradient in order to balance the learning rate for each weight of the model network. Due to the increased cost and lower performance of Adam’s algorithm a better performing SGD algorithm is proposed.
Stochastic Gradient Descent can be proposed instead of the usage of the typical Adam’s algorithm, it promises greater optimization for the GCN while adhering to all the regular layers of the same. SGD finds out the gradient of the cost function that is chosen in each iteration for the results generated in FAIGCN. It has a comparatively shorter training time than other Gradient Descent Algorithms.
Previous research states that it is difficult to use batch gradient optimization algorithm which involves iterating over the entire set of nodes and then calculating the cost function, \[\label{GrindEQ__7_} \mathrm{\nabla }L=\frac{1}{|V_L|}\sum\limits_{v\in v_L}{\mathrm{\nabla }f(y_v,z^{(L))}_v)} , \tag{7}\] this is significantly improved via the usage of stochastic function which uses a mini batch which is calculated by performing sampling over the data. \[\label{GrindEQ__8_} \frac{1}{|V_B|}\sum\limits_{v\in v_B}{\mathrm{\nabla }f(y_v,z^{(L))}_v)} . \tag{8}\]
For the proposed FAIGCN network the Stochastic Gradient Descent with Variance Reduction could be applied as mentioned below: 1. Using random sampling consider a mini batch of nodes where \(v_B\in v_L\). 2. Develop a graph of the activations h(v)l and h (v)l that would be used for the mini batch in each iteration. 3. Use forward propagation to derive the predictions. 4. Use backward propagation to generate the gradients and parameters are recalculated using SGD. 5. Update the activations used previously.
It was generated by deployment of the synthetic Skinned Multi-Infant (SMIL) Model [9] to the video sequences recorded on real-world infants from a hospital. The dataset accounts for a total of twelve RGB D sequences of babies of about 1 and a half years old. This dataset is open- sourced and ready to be used for any research experiments to be conducted on the same. The rendered images are of enhanced shape and movement along with the texture demonstrated to depict the real-life infants, by replacing the real, raw images by the rendered images of the infants as depicted in Figure 2. The annotation of each video is also provided to signify the presence or absence of agitated movements of the infants to demonstrate if a certain image is classified to be normal or abnormal by the guidance of a medical expert using the GMA method [9].
RVI-38 data was sourced as a part of clinical care routine at the Royal Victoria Infirmary (RVI) in Newcastle upon Tyne, UK. The data is collected from babies between 36-60 weeks old, there are 38 RGB-D video sequences with each having a resolution of 1980×1080 with a 25FPS frame rate.
Each video is in the range of about 5 minutes and 40 seconds long with the average being about 3 minutes and 36 seconds. These videos were annotated by the GMA method as well, similar to the MINI-RDB-D.
Xavier Institute of Engineering provides the only GAIT lab in India that can provide highly subsidized motion assessment. This lab provides variety of equipment and tools for human motion analysis. Students of engineering field can make use of the same for working on clinical technology and use the data and the generated results for their research. GAIT lab equipment was used to collect data for Indian adult patients suffering from cerebral palsy to find an adequate threshold and the respective testing and training data for analysis.
This is the structural analysis of human or animal movements, especially walking. It is difficult for software to analyze complex human motions such as walking. Gait analysis assists in scrutinizing walking sequences [3, 10]. The state-of-art motion capture tool helps to study these sequences and detect the cause of the disability. This further provides aid in designing the treatment plan and therapy []. It uses image processing approaches that include cameras which have the capability of capturing the gait parameters from various angles. On the other hand the sensor approaches gather information related to gait and are connected to the person [8, 7].
This provides the tools that are necessary for complicated bio-mechanical modeling, kinematics and temporal analysis [12], and their reporting. It calculates metrics using the given 12 parameters:
1) Pelvic Tilt
2) Hip Flexion/Extension
3) Knee Flexion/Extension
4) Ankle Flexion/Extension
5) Pelvic Obliquity
6) Hip Abduction
7) Knee Valgus
8) Foot Progression
9) Pelvic Rotation
10) Hip Rotation
11) Tibial Rotation
12) Foot Rotation
A library that detects various key-points of the human body such as hand, face, foot and so on. (Together detects 135 key points). It is a pose estimation method that is used for real-time multi-person keypoint detection. Fundamentally a human pose skeleton is considered as a set of data points which when connected depict an individual’s pose. Every data point is considered as a coordinate and the connections between these coordinates are called limbs. It is a robust and structured approach that assists in applying these pose estimation methods to images irrespective of their background, including crowded scenes. It has proven to be efficient due to features like real-time 2D–multi and 3D- single person keypoint detection [1]. It supports video input from camera streams, images, videos and so on. It can be run on various operating systems like Mac,Windows,Ubuntu and it supports Nvidia GPU, AMD GPU and non-GPU or CPU. OpenPose works by extracting the features from an image, then by putting these features into the two-branch convolutional network layers. These branches predict certain fields that depict the relations between parts. The further stages then clean these results and allocate the pose skeletons.
The system flowchart is shown in Figure 3 and the process flow novelty of the paper is shown in Figure 4.
The previously implemented systems have used the bench mark datasets and standard results using the same have been provided, but due to the limitations of the dataset, usage of the collected dataset from the GAIT lab is being made for the implementation of the model. This would make the model compatible and allow the it to provide adequate performance on the Indian audience that would be using the system.
Apart from the standard changes in the dataset, the project aims to serve the research gap between the usage and compatibility of the project with a different optimizer instead of the standard proposed Adam’s optimizer, Stochastic Gradient Descent optimizer was used which is expected to provide better performance and efficiency as compared to the previously mentioned optimizer.
A conclusion based on the test results would be drawn for an efficient comparative study of the same.
Gathering of the results from the GAIT lab was successfully done, the results demonstrated the various coordinates of the human body which demonstrate the joints (Figure 5).
Comparison of this CNN algorithm with other ANN and RNN would be done to find the best algorithm for the study and implementation.
Execution of FAIGCN model on the RGB-D and RVI dataset was successfully completed. The results with accuracy with the test dataset was displayed and the experiment was completed successfully.
Extraction of the coordinates from the MINI-RGBD dataset was done and the coordinates were collected from the same, for processing to detect the presence of Cerebral Palsy in the infants. The coordinates obtained represented the 3-D structure as well as the 2-D structure of the infant’s body (Figure 6).
The program for execution of the Open Pose model was successfully implemented and information was collected from the same (Figure 7).
Study of various State-of-Arts model was successfully completed, of which accuracy in the model of FAIGCN was highest and hence chosen.
Implementation of the model for FAIGCN using the mentioned layer (32, 32) and (32, 69) (Figure 4).
Visualization of the results using CUDA-GEFORCE NVIDIA is ongoing.
The implementation of a modified model for the GAIT lab test results is under implementation.
The training process produced multiple outputs at different stages, as shown in Figures 8, 9, and 10.
Further, knee-flex parameter variations were studied across multiple stages, shown in Figures 11, 12, 13, 14, and 15.
The comparison of accuracy and mean loss values after multiple epochs is shown in Table 1, where the model demonstrates progressive improvement with training.
Analysis of the GAIT lab data with Time Series using the ARIMA model was completed to allow visualization of individual data successfully, complementing the accuracy results reported in Table 1. The ARIMA-based approach enabled the decomposition of the raw GAIT signals into trend, seasonal, and residual components, which provided deeper insights into the underlying patterns of movement across different subjects. This modeling not only confirmed the stability of the extracted coordinates but also highlighted fluctuations that were otherwise difficult to detect through conventional accuracy metrics. Furthermore, the application of ARIMA to time series data facilitated the identification of anomalies and subtle variations in joint motion parameters, which directly supported the comparative analysis of the CNN, ANN, and FAIGCN models. The combination of accuracy results (Table 1) and the visualized ARIMA outputs thus offered a comprehensive validation framework, ensuring that both predictive performance and temporal dynamics were effectively captured for clinical interpretation.
| Epochs | Accuracy | Mean Loss |
|---|---|---|
| 100 | 97.9845 | 0.6850 |
| 200 | 98.4565 | 0.6800 |
| 300 | 99.2647 | 0.6755 |
| 400 | 99.8493 | 0.6718 |
| 500 | 100.0000 | 0.6690 |
Cerebral palsy (CP) is a group of disorders that can involve the brain. This affects nervous system functions, such as movement, learning, hearing, seeing, and thinking. There are several different types of CP, including spastic, dyskinetic, ataxic, hypotonic, etc. Each of these four types affects the body in different ways. Depending on the type of CP the muscles in the person’s limbs may be too stiff or too loose. The main objective was to build a machine learning model and test the accuracy of the model implemented in order for the identification of Cerebral Palsy (CP) and further detect how severe it can be. A lot of studies have been conducted on the same but to evaluate Cerebral Palsy in static data i.e. data that is not real time. The model hence collected real time data images and further converted into co-ordinates and performed the following evaluation on the dataset extracted.