Android Fall Detection System for construction workers

Jane Chung

Magor Kátay

Tijs van Lieshout

Mitsy Prada

Tjitte Talsma

26/06/2019

Abstract

Falling is the most common type of injury for construction workers. When a fall has taken effect, it can be difficult for others to notice on time. The victim of the fall might not be able to contact others to ask for help. By making a fall detection system a phone application, the construction worker avoids carrying additional equipment. Although there are plenty of options for iOS, the choices on Android are lacking. Here we present Android Fall Detection System (AFDS): an application written in Python to detect and notify others of a fall happening in a construction environment. AFDS works by using the accelerometer to detect a fall in three phases: free fall, impact and the lack of movement. After a fall has been detected, a chosen phone contact gets notified by means of a call and GPS location through SMS. We show that the algorithm for the detection of falls above half a meter has an accuracy of 92% with n=100. Contrarily the separation of daily activities of construction workers (true negatives) with falls (false positives) has an identification accuracy of 95% with n=100. Although these tests have proven to be reliable the validity has to be taken in question. There is still a need for further tests that more truthfully simulate a falling person. Also noted is the need of improvement in the usability of the application in the form of making the fall detection proceed while the phone is locked.

Table of contents

1. Introduction
   1. Background knowledge and current situation
      1. Fall detection with Apple Watch Series 4
      2. Workers safety pro functionality
   2. Problem definition
   3. Goal of the project
2. Procedure
   1. Writing an Android application in Python with the Kivy library
      1. Architectural pattern of the code
      2. Detecting a fall with the accelerometer
      3. Notifying that a fall has happened
      4. Saving a phone number to call for help
   2. Building an Android application package with Buildozer
3. Results
   1. Functionality of Android Fall Detection System
   2. Testing of Android Fall Detection System
4. Discussion
5. Conclusion
6. Literature
7. Appendices
   • Appendix A: Mobile Operating System Market Share Netherlands
   • Appendix B: Requirements
   • Appendix C: Supported Application Programming Interfaces of features accessible through Plyer
   • Appendix D: Source code of Android Fall Detection System
   • Appendix E: GPS visualization map
   • Appendix F: Measurements accelerometer

1. Introduction

1.1. Background knowledge and current situation

Construction workers are exposed to severe hazards. The most serious hazards these construction workers are exposed to include: falls, being struck by an object, electrocutions, and being caught in between two objects. (Occupational Safety and Health Administration, 2017).

Current regulations regarding fall protection state that employers are responsible for providing fall protection. This fall protection can be provided by guardrails, safety nets, personal fall arrest systems, positioning device systems, and warning line systems (Occupational Safety and Health Administration, 2016).

Despite European and Dutch norms and regulations, construction workers are exposed to possible falls. This represents an important hazard, therefore there are specific norms to prevent accidents to happen. One possible additional approach is providing a fall detection system that provides an alarm and contacts the appropriate person in case of an emergency. For the target group previously mentioned, currently the market provides the following products:

1.1.1 Fall detection with Apple Watch Series 4

• Functionality: It detects a hard fall (when wearing it).
• After detecting a hard fall, you choose to call emergency services or not in case it detects that you are moving.
• If you are not moving during one minute after the fall, it calls emergency services automatically. After the call, it sends messages to your emergency contacts, and begins a 30-second countdown, while tapping you on the wrist and sounding an alert. The alert gets louder, so that you or someone nearby can hear it.
• In your iPhone you can turn off and on the fall detection system, and you can add you medical ID and emergency contacts. There is no way to pair an Apple watch with an Android phone.
• Cost: 429€ (Apple Watch Series 4).

1.1.2. Workers safety pro functionality

• When there is a fall detection it starts a countdown. If the countdown is not stopped, an alarm goes off and the message is sent to the contacts. A safety dashboard enables centralized monitoring, provisioning and response. The time of the countdown is customizable. The application does not contact emergency services. Workers Safety Pro is available for iPhone but not for Android devices.
• Cost: 8.84€/month (application for iPhone)

1.2. Problem definition

Available products in the market intended for the aforementioned target group are limited, and they work only in Apple devices. These products are relatively expensive and have limited functionality considering that most people in the Netherlands uses Android as mobile operating system (StatCounter, 2019) (Appendix A).

1.3. Goal of the project

The goal of this project is to create a fall detection system that detects falls of construction workers by using two sensors and sending a signal to another device, which receives and reads the signal in case of an emergency. The main goal is to detect falls within an acceptable level of accuracy, keeping both the number of false negatives and false positives as low as possible. The prototype cannot exceed the cost of 50 euros, it needs to be wearable and ergonomic, and its battery needs to be rechargeable (Appendix B).

Construction workers already carry a lot of equipment for their day-to-day tasks. Adding even more devices to carry would be inconvenient or even dangerous. However, what this group of people often do carry outside of their work-related tools is their personal mobile phone. Therefore, it has been decided to design and develop a sensor system as an android application that uses the sensors within a mobile phone running on android to detect falls of construction workers. A lower height limit of half a metre has been set as a minimum to detect falls.

Here we present the Android Fall Detection System (AFDS). AFDS uses the accelerometer and GPS of an android phone to detect and locate falls of construction workers. When a fall has been detected the application informs a previously added contact that a construction worker may need help by making a phone call and sending an SMS containing the location of the fall to the contact.

2. Procedure

2.1. Writing an Android application in Python with the Kivy library

Android Fall Detection System was written in Python 2.7.13 with help of the Kivy library (Kivy.org, 2019). Kivy is an open source library for Python to develop cross platform applications for Linux, Windows, OS X, Android, iOS and Raspberry Pi. Kivy includes an Application Programming Interface (API) called Plyer (Plyer.readthedocs.io, 2019). Plyer was used to access sensors and other features of the android phone such as calling and SMS (Appendix C). The chosen features include the use of the accelerometer for detecting falls, the GPS for locating where a worker has fallen and SMS and calling for contacting help. The source code of Android Fall Detection System is available under Appendix D.

2.1.1. Architectural pattern of the code

The Python code of the application is split up into four classes in which each class represents a screen. By using the kv design language the User Interface is split up into a main, contacts, accelerometer and timer screen (Kivy.org, 2019). In this kv-file all the graphical elements are defined to adopt to the separation of concerns principle (Laplante, 2007).

2.1.2. Detecting a fall with the accelerometer

The fall is detected by the accelerometer, the detection of the fall is determined in three stages. The first stage is when the G-force of the device is lower than 0.1 for more than 319 milliseconds, this means that the device is in freefall from a height of half a metre (Formula 1). The second stage is completed when the accelerometer measures a force around 1,5G, this means the device has had an impact with a surface. For the last stage the accelerometer measures if the G-force is between 0,75G and 1,5G. If the accelerometer measures values outside of these thresholds it is deduced that the person is still moving or getting up resulting in the algorithm concluding that the fall was only minor. The previously mentioned thresholds have been based on literature of reading the values of the accelerometer in free fall (Sposaro and Tyson, 2009) (Appendix F).

Formula 1: Time t taken for an object to fall distance d. This formula was used to calculate that a fall of half a metre takes roughly 319 milliseconds.

2.1.3. Notifying that a fall has happened

After the fall detection algorithm concludes a fall has taken place the timer code activates. This code counts down from 30 seconds in which every five seconds a Waveform Audio File (WAV) is played asking the construction worker to click a button confirming that he does not need help. If the construction worker fails to do this within 25 seconds a second audio clip is played, and the GPS of the android phone is turned on to start calibrating the exact location of the construction worker. When the timer reaches zero the location of the construction worker is finalized and gets send as a Google maps location to a phone number that the construction worker has specified under the contacts section of the application. This phone number will also be called to both notify and communicate with the helping party.

2.1.4. Saving a phone number to call for help

To save phone numbers into memory the Python csv library was used. This library provides the ability to write and read text to csv-files (comma-separated values) such that when the application is closed the phone number is still saved. When the user enters a phone number it gets written to a csv-file hidden in its application folder. Once the contacts screen is loaded the csv-file is read and the phone number is being shown to the user.

2.2. Building an Android application package with Buildozer

Mobile applications on the Android operating system are often programmed in programming languages such as Java, Kotlin or C++. To be able to create an Android application package (APK) from Python code Buildozer was used (Buildozer.readthedocs.io, 2019). Buildozer is a tool from Kivy that automates the python-for-android compiler with help of a buildozer.spec file (Python-for-android.readthedocs.io, 2019). Although the Kivy developer team has made no official statement it has been determined that python-for-android, and thus Buildozer, has no windows support at time of writing. To solve this limitation a virtual machine running Xubuntu 17.04 has been used to compile the Python source code into compatible code for android (Xubuntu.org, 2019). To be able to detect android phones inside the virtual machine it was needed to install the Android Debug Bridge (adb) (Android Developers, 2019).

3. Results

This section of the technical report aims to show the results of this project, starting with a short explanation of the functionality of the application and then providing a test of its accuracy. AFDS (Android Fall Detection System) allows the user to start/stop detecting falls and add an emergency contact that would be contacted if a fall is detected and the user does not stop the alarm.

3.1. Functionality of Android Fall Detection System

Figure 1: Android Fall Detection System

After opening the application (Figure 1) the user has access to the main menu (Figure 2). “Edit Contacts” allows the user to add an emergency contact through the contact screen (Figure 3). The “Back” button in this screen makes the application show the main screen again, from where the user can start or stop the fall detection system. To start detecting falls the user can press this button, which leads to the fall detecting screen. From this screen the user can go back to the main menu with the “back” button (Figure 4). In case a fall is detected a countdown is activated, as it can be seen in the fall detection screen (Figure 5). If the construction worker needs no assistance, the button “I am okay” stops the countdown. If this button is not pressed within thirty seconds, the application makes a call to the emergency contact (Figure 8) and sends an SMS with the location of the fallen construction worker (Figure 7).

Figure 2: Main screen of AFDS

Figure 3: Contact screen android AFDS

Figure 4: Detecting fall screen android AFDS

Figure 5: Detected fall screen android AFDS

Figure 6: Cancelled countdown screen android AFDS

Figure 7: Missed call after fall detection

Figure 8: SMS sent after fall detection

3.2. Testing of Android Fall Detection System

The GPS test has been performed in three different environments to test the difference in accuracy between inside and outside (Table 1). A Samsung Galaxy S9 was used in all GPS tests. The data has been split into the degree of longitude and latitude and the test was performed in Assen, The Netherlands (53,00°, 6,57°). In these tests the manually curated location of the phone, which has been deduced as the most accurate estimation, is referred to as the true location. Each guess is what the Android Fall Detection System chooses as the location of the fall. The errors of the true and guessed locations with the average errors have been calculated and are shown in Table 2 and Table 3. To be able to interpret this data more easily, the errors have been converted from degrees latitude and longitude into meters (Table 4) (Veness, 2019).

To better visualize the errors of the GPS both the true and guessed locations have been plotted onto a satellite image of the building in which the tests have been run (Figure 9a and 9b).

Table 1: Longitude and latitude given by the GPS of three different locations compared to the true location.

Table 2: Errors between the true and the guessed locations based on the GPS accuracy test. The table states the absolute differences between each guess and the true value of the three determined locations. Also included is the average error per location.

Table 3: The total average of all errors of all locations.

Table 4: Errors between the true and guess locations based on the GPS accuracy test in meters. The Haversine formula has been used to calculate the great-circle distance between two GPS points. The great-circle distance is the shortest distance over the Earth’s surface.

Figure 9a: True and guessed locations plotted on a satellite image. The locations shown as a star icon are the true locations and the marker icons represent the guessed locations. The icons highlighted in orange represent tests done outside, the blue icons represent tests done under a roof and icons in green represent tests done under two ceilings and a roof.

Figure 9b: Zoomed in version of Figure 9a

To test the fall detection algorithm, both true falls that should be detected and false falls that represent daily activities of a construction worker have been tested (Table 5 and Table 6). A Samsung Galaxy S7 was used in all fall detection tests. The testing of true fall detection was performed by letting the phone fall from a specified height. To protect the phone, a protective case and a landing platform consisting out of a summer jacket was used. In order to test the daily activities, a test subject placed the phone into their front pocket while performing said activities. All tests were categorized depending on whether the falls have been detected or not (Table 7).

Table 5: Detection of true falls from different heights.

Table 6: Detection of false falls while performing activities.

Table 7: Overall accuracy of fall detection.

To be able to test what a typical freefall would look like the values of the accelerometer in the android phone were read (Figure 10). Based on these values and literature it has been concluded that a fall consists of three phases (Sposaro and Tyson, 2009). The first phase can be seen as a freefall, in which the values go below 1 m/s2. Following a freefall is an impact with the ground, causing the total acceleration to spike to well over 15 m/s2. The third phase is the detection of movement. If the total acceleration goes outside of 6 and 15 m/s2 for more than two and a half seconds it is concluded that significant motion has taken place and the user is okay.

Figure 10: Total acceleration over time of a free fall of one meter. Phone was let go from one meter and landed onto a blanket. The total acceleration of the X, Y and Z axis combined have been plotted in m/s2.

4. Discussion

Android Fall Detection System can be used to detect falls of construction workers and contact help if needed. AFDS does however come with some disadvantages.

Because of the need of the Kivy Buildozer to create Android-compatible code, AFDS has been limited to Python 2.7. This limits the maintainability of the source code since Python 2.7 will be deprecated in the beginning of 2020 (Python.org, 2019). AFDS could also benefit from some extra features such as a back button after stopping the countdown and the ability to add multiple phone numbers in case the first person is not available. The most necessary feature AFDS is missing is arguably the ability to keep the fall detection algorithm running while the phone is locked.

Part of the Android Fall Detection System is to send the location of the fall to get help. This location is determined by the GPS of the phone (Table 1, Table 2 and Table 3). As seen in Figure 9a and Figure 9b the results significantly vary based on the environment of the fall.

In the case of a fall happening in an area without a roof the guessed locations seem to be the most accurate of the three different locations. However, the precision is less than under a single roof. This could have been by stochastic error since the sample size of these tests were limited to three.

Although the tests under a single roof are the most precise, they seem to be the most inaccurate. The hypothesis is that this is caused by the many windows of this location with the reflection causing the GPS location to be moved up north.

As expected, the tests under two ceilings and a roof seem to be both somewhat inaccurate and not precise. The environment of this location is excluded from the open air which makes determining the GPS location more difficult.

The test to see whether the Android Fall Detection System can detect true or false falls has been carried out. When referring to Table 4, fall detection at variable heights impacts the accuracy of the algorithm. The application has detected falls 65% of the time at half a metre with a sample size of twenty. However, the accuracy was at 98.75% from one to two and a half metres which is significantly higher than at half a metre. Thus, it has been concluded that a fall above at least half a metre significantly improves the accuracy of the fall detection.

Another test has been performed to detect if daily activities of a construction worker would set off the fall detection algorithm (Table 5). Five activities that could be misunderstood by the algorithm as a fall, which is referred to as a ‘false fall’, have been performed. These tests have been conducted twenty times each for a total of one hundred trials to see how accurate the Android Fall Detection System is in distinguishing true negatives from false positives. When the phone was dropped from 0,4 metre, the application identified the drop as a fall 25% of the time resulting in a 75% accuracy. In the other activities the application did not give a false conclusion giving a 100% accuracy.

The conditions of testing the accelerometer were however less than ideal. All tests were performed by letting the phone fall freely onto a summer jacket to lessen the impact. This could have impacted the resulting data and made the results less valid. In an optimal test it would be best to have a human or test dummy fall from the specified heights onto a concrete floor with the phone being placed in the pocket of some pants.

Universally a fall consists of a period of low acceleration while falling and a spike in high acceleration at impact. Sposaro and Tyson, 2009 describe a typical fall to have these low values under 0,6G and high values above 3G (Figure 11). However, in said paper a fall is defined as a standing person falling backwards onto a surface of the same height as they are standing on. Since Android Fall Detection System defines a fall as a free fall above at least half a metre the thresholds diverge from this paper. The algorithm described in this report uses a lower threshold of 0,1G for more than 319 milliseconds with an upper threshold of at least 1,5G.

Figure 11: Total acceleration over time of a typical standing fall. An example of total acceleration values of a backwards fall of a person that was standing. Based on these falls the deduced thresholds are around 0.6G and 3G. Graph adapted from (Sposaro and Tyson, 2009).

5. Conclusion

Within our test conditions Android Fall Detection System has been deemed accurate enough to reliably detect falls of construction workers. Further testing is however needed to make a definite conclusion about the appliance of AFDS in a real-world setting. For further research it would be beneficial to test the fall detection algorithm by letting a human or test dummy fall on a more typical work floor such as one made from concrete.

The locations found by the GPS have been regarded to be accurate enough to locate where the fall has happened. The sending of the location to another phone number has proven to be reliable. It can be said that GPS is not an ideal way of locating a fall since construction sites often have multiple floors and GPS does not detect height.

A major downside of this version of the application is that the detection of falls is paused when the phone is locked. Having the phone unlocked in their pocket can cause the user to accidently press other buttons while working and make the app not detect falls anymore. It is therefore recommended that this is at the top of the list of features to work on in further development.

The use of Kivy and Buildozer to create Android applications from Python code is troublesome and time-consuming. Because of poor documentation and many outdated dependencies, it is not recommended to use these libraries and a better alternative is to write the code directly in Java.

6. Literature

Android Developers. (2019). Android Debug Bridge (adb) Android Developers. [online] Available at: https://developer.android.com/studio/command-line/adb [Accessed 19 Jun. 2019].

Apple Watch Series 4. (2019). Apple (América Latina). Retrieved 9 May 2019, from https://www.apple.com/la/apple-watch-series-4/

Buildozer.readthedocs.io. (2019). Welcome to Buildozer’s documentation! — Buildozer 0.11 documentation. [online] Available at: https://buildozer.readthedocs.io/en/latest/ [Accessed 19 Jun. 2019].

Veness, C. (2019). Calculate distance and bearing between two Latitude/Longitude points using haversine formula in JavaScript. [online] Movable-type.co.uk. Available at: https://www.movable-type.co.uk/scripts/latlong.html [Accessed 26 Jun. 2019].

Distribution dashboard Android Developers. (2019). Android Developers. Retrieved 9 May 2019, from https://developer.android.com/about/dashboard

Kivy on Android — Kivy 1.10.1 documentation. (2019). Kivy.org. Retrieved 9 May 2019, from https://kivy.org/doc/stable/guide/android.htm

Kivy.org. (2019). Kv Design Language — Kivy 1.11.0 documentation. [online] Available at: https://kivy.org/doc/stable/gettingstarted/rules.html [Accessed 19 Jun. 2019].

Kivy.org. (2019). Welcome to Kivy — Kivy 1.11.0 documentation. [online] Available at: https://kivy.org/doc/stable/ [Accessed 19 Jun. 2019].

Laplante, P. (2007). What Every Engineer Should Know about Software Engineering.

Mobile Operating System Market Share Netherlands StatCounter Global Stats. (2019). StatCounter Global Stats. Retrieved 18 June 2019, from http://gs.statcounter.com/os-market-share/mobile/netherlands

Python.org. (2019). PEP 373 – Python 2.7 Release Schedule. [online] Available at: https://www.python.org/dev/peps/pep-0373/ [Accessed 20 Jun. 2019].

Plyer.readthedocs.io. (2019). Welcome to Plyer — Plyer 1.4.1.dev0 documentation. [online] Available at: https://plyer.readthedocs.io/en/latest/ [Accessed 19 Jun. 2019].

Python-for-android.readthedocs.io. (2019). python-for-android — python-for-android 0.1 documentation. [online] Available at: https://python-for-android.readthedocs.io/en/latest/ [Accessed 19 Jun. 2019].

Raadgevendbureauspeelman. (2019). Raadgevendbureauspeelman.nl. Retrieved 9 May 2019, from https://www.raadgevendbureauspeelman.nl/wp-content/uploads/2016/05/1.1-Cursusboek-VCA-versie-8.pdf

Sposaro, F. and Tyson, G. (2009). iFall: An android application for fall monitoring and response. 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

Welcome to Plyer — Plyer 1.4.1.dev0 documentation. (2019). Plyer.readthedocs.io. Retrieved 9 May 2019, from https://plyer.readthedocs.io/en/latest/

WorkerSafety Pro — Protects, detects, & alerts organizations automatically. (2019). Health & Safety solutions for Work and for Home. Retrieved 9 May 2019, from

Xubuntu.org. (2019). Xubuntu 17.04 released! « Xubuntu. [online] Available at: https://xubuntu.org/news/xubuntu-17-04-release/ [Accessed 19 Jun. 2019].

7. Appendices

Appendix A: Mobile Operating System Market share Netherlands

Appendix B: Requirements

• Sensor System for a specific target group
• Uses at least two sensors
• Sensor System sends a signal to another device in case a fall is detected. The signal is received and read
• Sensor System detects falls accurately
• Sensor System has a rechargeable battery
• Sensor System is wearable and ergonomic
• Sensor System prototype doesn’t exceed the budget of 50 euros
• Sensor System is finished before 21/06/2019

Appendix C: Supported Application Programming Interfaces of features accessible through Plyer

Highlighted in green are APIs that could be interesting for a fall detection system for construction workers that sends a signal to their foreman or colleagues. Edited from https://github.com/kivy/plyer.

Appendix D: Source code of Android Fall Detection System

https://github.com/Magiduck/AFDS2

Appendix E: GPS visualization map

https://drive.google.com/open?id=1OglZ2_CQCEmmW9vKLWWSUKcVu_fdZfcH&usp=sharing

Appendix F: Measurements accelerometer