Alexa voice integration
Technology comes along, and often it's used to make some business more money, but let's use it to solve a real pain point. Often it's hard to think of what those really helpful aspects are unless you are faced with a challenge.
The importance of data
Bringing infants home from the NICU means they are out of the worst, you hope, but how do you know?
When you goto the doctor they ask a lot of questions:
- How often are diapers getting changed and what's in them?
- How often are they eating, and how much when they eat?
- How high are their fevers, how long do they last, and how often?
- When have these things been done by you or the other parent?
You've barely been sleeping, let alone staying 100% connected across all the days, it's exceedingly hard to know the answers to these questions.
The solution of course is keeping track of the data and then analyzing it for trends.
Recommendations in the market
There's a series of recommendations out there, but they all face challenges:
Use a notebook, but if it's not around or you can't find it at 3am, you don't track the data
Use an app, but there's not apps that track all the information that you might want to track, so then you lose those data points.
Use an online spreadsheet to collect the data, but its not great on a mobile device and you aren't going to pull out a laptop for every data point log.
We went with a Google spreadheet, it at least wasn't stuck in the other room and could collect whatever data points we decided to track.
What if you could delegate the logging of this data to someone else?
We asked this while changing a diaper in the middle of the night: Why not Alexa (or Google Home)?
We didn't want to bring a solution to the market, we knew different people would have different challenges and would collect different data, but we used this to experiment with creating voice skills and the APIs to integrate with them.
This also had the added benefit of being able to log data while you were doing a feeding or a diaper change or a fever check and hands free because often your hands were... full.
Creating a custom skill
By keeping a skill in development mode you can test it on your own linked Alexa devices without releasing it which allowed us to cut corners on authentication and sheet selection and just focus on the API connection to write data to the sheet.
We could always add those enterprise aspects in later if we need to.
The custom skill was an API that took data and wrote it to a hard coded Google Sheet with hard coded columns. We then hooekd up the API to the conversational model to turn speech to text into the inputs for the API calls.
The conversational model
At first we created conversational models that alloed for back and forth.
Nate: Alexa, log a diaper change
Alexa: For who?
Nate: For Baby 1
Alexa: How full was it?
Nate: 3
Alexa: Was there Poop?
Nate: No
Alexa: Diaper logged on January 9th at 3:11am, medium full with no poop.
When gathering data we found that this was waaaay to time consuming, so we created shortcut dialog:
Nate: Alexa, log a diaper change for baby 1 will fullness of 3 and no poop
Alexa: Diaper logged on January 9th at 3:12am
With many data points coming in throughout the day it made it worth the long winded single shot message.
We collected 4300+ data points over 6 months.
Data was across 2,107 feedings, 2,147 diaper changes, and 48 fever temperature events.
We averaged 24 data points per day.
What did we do with the data?
We could ask Alexa for the last time there was a diaper change or what the last feeding time was.
We could now answer how long it's been since a poopie diaper and how much prune juice they've had to help.
When the doctor checkups asked questions, we were armed to know
When we were worried about fevers or odd symptoms, we could use the data to see see most cases were anomalies and ride them out instead of heading to an ER.
We made charts to visually see the trends we were mining the data for.
We could see how they were recovering from their preme status against the common weight and heights for their age groups.
Unexpectedly, it also let us perform projections on diaper and formula usage so we could plan shopping/budget when we needed to get more from the store. Seemingly not important, it let us optimize what few hours we had to catch up on sleep instead of wandering around a store more often than needed.
The conversations were limited to the data analysis and the conversational models created. In the GenAI context this could only have been made stronger by a GenAI agent.
Interested more about big data? Check out The Human Face of Big Data
Recent lab experimentation (since 2020)
Lab descriptions and details currently not available online, please reach out to discuss.
