A traditional computer mouse provides four main functions to a user–
Ability to precisely move the mouse pointer in real-time
Ability to click (left or right or double)
Ability to drag
Ability to scroll
A hands-free mouse can be simply defined as any technology (hardware and/or software) that allows a user to achieve all the computer mouse functions without having to use a hand or any part of it.
Many commercially available “hands-free mouse” products as well as voice control solutions fail this simple test. Therefore we will not be including them in this review article.
How Does A Hands-free Mouse Work?
Most hands-free mouse solutions involve use of a moveable body part as an alternative to the hand, in order to move the mouse pointer. Some examples of alternative body parts are head, chin, mouth, and eyes. Motion of these body parts are sensed using some type of motion sensors and those detected motions are then translated into mouse pointer motions. Additional user actions such as holding still (dwelling), making a facial gesture, sipping or puffing, biting, pressing a switch with a foot, chin, elbow, etc. are used to perform clicking, dragging, and scrolling functions.
See below for description of various types of hands free mouse solutions, how they work, and their pros and cons.
Types of Hands-free Mouse Solutions
1. Wearable Motion Trackers
These types of devices look like a head-band, an arm-band, or even eye wear. You wear this mouse accessory on your body and typically connect with your computer using a wireless connection. Motion sensors in these devices then detect any motion made by the user, and then translate them into corresponding mouse pointer movements, much like a computer mouse.
Wearable motion trackers are great for users who have a reasonable level of head, foot or arm control. These devices also provide the user the ability to move around in a large area while controlling their computer, without being in the line of sight with it. However, they may not be suitable for people with extreme mobility impairments as they still rely on some degree of physical movement of a body part.
One big disadvantage with these devices is that they are incomplete without a convenient way to click, drag, and scroll the mouse. This means that the user also needs to purchase and install some type of a switch accessory to achieve clicking, dragging or scrolling functions; e.g. a bite switch– operated by biting on it, a sip-and-puff switch– operated by sipping or puffing into it, or a foot switch operated with a foot, and so on. Bite switches can be especially cumbersome since the user needs to physically put them in their mouth, thereby causing awkwardness as well as sanitary issues.
To get around the above additional hardware requirements, many wearable motion trackers have to rely on the “dwell clicking” mechanism. Dwell clicking means that every time the user stops moving the mouse, that results in a click after a set amount of minimum time. While this helps with ease of clicking, it can lead to frequent unintentional clicks, which can be irritating and be a hindrance to productivity. This forces the users to continuously manage it by turning the clicking function on or off. Use of dwell clicking also means that to achieve additional functions such as right clicking, double clicking, dragging, and scrolling, the user has to perform additional steps on the user interface to manage the intended result of a dwelling action.
2. Lip/Mouth/Chin Operated Joysticks
As the name suggests, you operate these joystick devices by using your lips, mouth or chin. You can then move the mouse pointer using your lips, mouth or chin. If you are using your lips or mouth then you need to hold part of the joystick inside your mouth. However, if you plan to use your chin, you need to place the joystick control just below your chin so that you can use it with chin motions.
Note that you must mount these joysticks on a stable surface or with a sturdy support, and they must be mounted such that the joystick is fairly close to your mouth during operation. Therefore, unless you mount one on your wheelchair, you will have to come close to the location of the joystick while you use your computer.
These devices can be a good choice for people with limited head mobility as the mouse pointer motion can be achieved with minimal mouth, lip or chin motion. However, these gadgets can take quite some time to master as they can be very sensitive to small motions of your mouth or chin.
As with wearable motion trackers discussed above, additional mouse functions can be achieved using dwell clicking or using additional mechanisms such as a sip-and-puff switch, mouth operated switches, foot switches, etc. This brings in additional complexity for the user in set up and training, as well as increases the cost of purchase and maintenance. Further, you will need to get used to the awkwardness of holding something in your mouth and potential sanitary issues. Having said that, gamers with disabilities can achieve a high level of proficiency with the use of joysticks along with multiple switches to play complex video games.
3. Reflective Dot Trackers
In these systems, you must wear a reflective dot on your forehead or something that you wear on your head (such as eyeglasses or a cap). A specialized infrared camera is mounted on your monitor to track the movements of the reflective dot as you move your head. These detected movements are then converted into mouse pointer movements on the screen.
This is a good choice for people who can move their head comfortably. There is no requirement to wear a gadget on the body, so that makes it convenient for the users, though you do have to mount the specialized camera on your monitor. The mouse pointer is easier to control than a joystick as the pointer motions are proportional to head motion.
Given that these devices come with a specialized camera, they can be expensive. However, they may be covered by health insurance when required for operating speech generation software. As with other methods discussed above, these devices also need to rely on dwell clicking or additional accessories for the purpose of mouse clicking, dragging, and scrolling. This causes an incurrence of additional costs and complexity of use. Further, just as with any hardware based systems, there can be breakages and therefore repair or replacement costs. You may also be required to purchase the reflective dots on a regular basis.
4. Eye Tracking Systems
These systems also involve the use of specialized, state-of-the-art eye tracking sensors. These eye tracking sensors are typically mounted just below your monitor. These sensors then track your eyes or pupils, more specifically, the location and orientation of your eyes. As you look around the screen, these systems then calculate the gaze point, that is, the location you are looking at on the screen. The mouse pointer is then transported to the calculated gaze point at any given moment in time.
Eye Tracking systems are ideal for people with severe mobility challenges, for example, people who may be completely paralyzed and who cannot move any body part (including their head or chin) in a controlled fashion. This type of solution does a great job of enabling people to communicate using Augmentative and Alternative Communication (AAC) speech-generating software.
A big disadvantage with these devices is that they are expensive. In some countries, such as the United States, such devices may be partially covered in cases a user needs them for speech generation purposes.
Eye tracking technology has inherent limitations in calculating the gaze point precisely, despite the performance of multiple calibrations. This means that it is harder to achieve pixel perfect pointer placement without having to perform extra steps (such as zooming in part of the graphic display) to refine cursor placement. Further, these systems also require the user’s eyes to be in a limited region / zone in relation to the eye tracking sensor (see “working box” in the figure), which can feel constraining to users.
As with other solutions above, eye tracking mice also have to rely on dwell clicking or other mechanisms for mouse clicking, thereby requiring additional accessories such as switches. Eye tracking systems also typically work only one display monitor. These factors are typically not much of an issue for people with severe disabilities and are primarily concerned more about communication using AAC / speech generation. However, above described factors may become a hindrance to folks more interested in achieving productivity, especially in professional settings.
5. Webcam-based Face Tracking Mouse Software
These software-only solutions are becoming popular thanks to their ease of use, convenience, and cost-effectiveness. The software includes advanced computer-vision AI algorithms to track the motions of the user’s head as well as their facial gestures via a built-in or a standard webcam. Head motions are translated to pointer motions, and facial gestures such as smiles, eye blinks/ squints, eyebrow raises, opening mouth, etc. are used for mouse clicking, dragging, and scrolling. This provides for a complete hands-free mouse solution without the need for purchasing or installing any hardware accessories, on or around the user’s body. This is a big advantage over other solutions.
Another advantage of this approach is that the user can download the software from the internet and start using it in a matter of minutes.
Software-only solutions also can provide multitude of customizable features, however, they may need porting to various operating systems.
Software solutions are ideal for people who want to explore affordable hands-free mouse solutions that are easy to use. Further, they can offer the easy and fluid experience necessary for productivity applications as well as gaming.
Introducing Smyle Mouse – A Novel Webcam Based Face Tracking Mouse Software
At this point, we would like to introduce a powerful, easy to use and affordable hands-free mouse solution that has not only won multiple industry awards but also has several patented features. Here are some key features:
Smyle Mouse software provides all the mouse functions out-of-the-box.
The software works with an in-built webcam in your Windows PC / laptop / tablet or any standard external webcam; no additional hardware required.
You can move the pointer with gentle head motion, even with limited range of head motions, and achieve pixel-perfect precision.
Clicking, dragging and scrolling is achieved with facial gestures such as smiling, opening mouth, raising eyebrows or eye blinks / squints, and even dwell clicking.
Smyle Mouse software provides a tremendous level of flexibility.
Use as a full-fledged mouse or just as an adaptive switch.
Use from a distance of 12 to 100 inches, and a very wide working area.
Users can use it sitting down, standing up or even when laying in their bed.
Use with multiple, large screen monitors.
Users can control all the settings on Smyle Mouse by themselves; no caregiver required from start to finish; you can even configure it to start automatically when the user logs in.
Plays nice with other computer access tools that the user may already have – joysticks, keyboards, voice-control, eye trackers, switches, and even traditional computer mice and touchpads! Simply turn on/off features on or off individually so that it can work with any existing tools that the user may already have.
Being a software-only solution, Smyle Mouse provides additional unique advantages over solutions that require specialized cameras, wearables, or any hardware widgets for that matter.
You can download Smyle Mouse from our website, install it, and start using it in a handful of minutes. No waiting for shipping and handling.
There are no hardware widgets to care for or to break and repair. No need to purchase supplies.
You can get 14-days of free trial.
Affordability and flexibility of financial commitment – Purchase a monthly or yearly subscription, or with a one-time payment.
In a nutshell, Smyle Mouse can be used by users looking for a simple head pointing solution for AAC/speech generation, or as a adaptive switch for use along with other pointing solutions such as eye trackers, or as a mouse pointing complement to voice-control solution, or a self-contained powerful mouse to enable one to compete at school or work completely hands-free.
How to Choose a Hands-free Mouse?
We believe that for a person with disabilities of hands, a hands-free mouse is the doorway to better quality of life via independent communication and productivity. Therefore, one should consider as many factors that are relevant to their situation. This can be an involved topic which warrants a complete article by itself. However, following are the top few considerations in choosing a hands-free mouse.
a. What are the physical limitations of the user?
Amount of physical control: How is your level and range of head control? Mouth? Facial expressions, and which ones? Vision issues? How many hours a day would you be using the system? In what orientations – sitting upright, or laying bed? In a wheelchair?
Verbal ability: Do you have any ability to speak? If yes, are you at least partially able to issue vocal commands to the computer?
Are the limitations temporary (such as due to a recent injury) versus expected to last for a longer term?
Would the user be comfortable continuously using the system for the number of hours they need to use it every day to achieve their goal? (See next factor below) This is especially important for wearable gadgets such as eyewear and headbands that some users may find uncomfortable to use for more than 30 minutes at a time.
b. What are the goals the user is trying to achieve via hands-free computer control?
For Communication / Speech generation only
Basic computer work – social media, internet surfing, email, environmental control, voice calls?
For school or college work? For office work to compete in a demanding professional environment?
c. Features and benefits necessary to achieve the user’s goals
Completeness of features:
Make sure the entire range of mouse functions are covered – moving pointer, clicking, dragging and scrolling.
Do you already use certain solutions but looking to complement them with a hands-free mouse? E.g. you are able to move the mouse reasonably, but now looking for a clicking, dragging, scrolling solution because you cannot use the buttons on the mouse? Or vice versa?
Ease of acquiring and trying the solution
How quickly can you acquire the solution?
Is it free to try? How long is the trial?
Convenience of installation and setup
Do you need any hardware widgets to be mounted / installed near the computer or on your wheelchair?
Do you need any gear to be worn on your body? If so, would you need a caregiver to help you wear it? What type of adjustments are needed to make sure it fits you well?
What settings need to be adjusted on the gear itself that will need help from a caregiver?
Ease of Use during operation:
Is the system easy to learn?
Is it reactive to your needs? E.g. Does the mouse pointer move easily, predictably and precisely based on your body motions? Does it click when you want it to? Are the functions achievable in a minimum number of actions, or does the system introduce additional steps to achieving compared to a standard mouse?
Does the system lead to unintentional results (such as unintentional clicks with dwell clicking), which you have to correct by doing additional work?
Are you able to completely operate the system independently, including various settings adjustments, without needing help from someone else?
If the system involves wearables, how many minutes or hours at a stretch can you comfortably wear it or use it?
Flexibility
Does the system play well with other tools which you may already have? E.g., you may already have some switches for mouse clicks, but now are looking for a mouse pointing solution? Or, you may already have voice control working well for typing and clicking but need a mouse pointing solution? How quickly can you switch between the two?
d. Affordability & Financial Flexibility
Total cost of ownership
What is the initial cost? If a hardware device is involved, what is the warranty cost? If something breaks in the hardware, what would be the cost to fix or replace?
What additional accessories are to be purchased? E.g. batteries, mounting gear, wires, USB hubs, etc.
Do you need to buy additional material in the future? E.g. reflective dots, bite switch covering, sip-and-puff mouthpieces, etc.
Financial flexibility
Does the product vendor provide flexibility in purchase? E.g. are monthly or yearly subscriptions available?
Health insurance coverage
Does health insurance cover any of the costs of the solution?
e. Product Support (Before and After purchase)
Does the solution provider company also provide product support? How is it provided? E.g. by email, phone or video conference?
Do they publish their contact information, including phone number?
Where is the company located? Are they available to speak in your time zone?
Conclusion
As we have seen above, each hands-free mouse has advantages and disadvantages, so you must choose one based on your specific needs and preferences. For people with extreme mobility impairments, Eye Trackers can be a great choice, as they can accurately track the smallest of movements in the eyes and convert them into real-time mouse motions. However, eye-trackers are expensive and do not offer the most fluid experience for people looking for high productivity or gaming. Lip/chin joysticks can be effective for productivity as well as gaming, however, they have a steep learning curve and need a lot of hardware set up around the user’s body. Reflective dot trackers and wearable motion trackers do not provide a seamless complete solution, such as for intentional clicking, dragging and scrolling, unless additional accessories are employed, which makes the whole setup even more complex and expensive. None of the hardware accessory based solutions are instantly acquirable, available to try for free, and purchasable on a subscription basis.
If you are looking for a cost-effective solution with flexibility and a wide range of features, webcam based face tracker software such as Smyle Mouse makes a great choice. It is easy to set up and use, with a very short learning curve. It is a great fit for people with a whole range of mobility challenges looking to regain their productivity at school, work, or home, playing video games, or even using speech generation software. Given that it can work with a built-in webcam, the user is not required to purchase and set up any additional hardware. Further, the innovative use of facial gestures provides a complete mouse replacement solution including mouse clicks, drags, as well as scrolls.
At Perceptive Devices LLC, we introduced the software-based face tracker, Smyle Mouse, way back in 2016. Since then, thanks to years of research and a focus on solving real pain points of people with mobility impairments, Smyle Mouse has won multiple awards and US Patents. Our software has undergone thorough testing and is easy to use for people of all ages.
Download our latest software and start using it in minutes. Use our 14-day free trial offer to personally experience the difference it can make in your life.
