Notebook showing usage of the data detective package.

To load from a local db, we just need the path

Alternatively, to load from a cloud database we load from a json file containing the url as the url contains our credentials and we want to lkeep these a secret. To learn how the Google CLoud SQL recorder can be setup checkout https://github.com/robmarkcole/HASS-Google-Cloud-SQL

Load the db data

#Load-the-db-data

We use the DataParser class to load data from the database. This class performs the SQL queries and parses the returned data. The class holds the master pandas dataframe master_df.

Loading output library...
Loading output library...
Loading output library...

Simple query

#Simple-query

Lets query a single sensor and demonstrate the data processing steps implemented by the library

Loading output library...

Helper to query by list

#Helper-to-query-by-list

Use fetch_data_by_list to query a list of numerical entities, must be from same domain and a minimum of 2 entities must be in the list. Returns a pandas dataframe.

Loading output library...
Loading output library...

Query all data

#Query-all-data

Data-detective takes care of parsing data from the database, intelligently sorting out numerical and categorical data and formatting them correctly. Use fetch_all_data to import all your db data into a pandas dataframe in memory -> this approach means it can take a while to load the data into memory, but subsequent processing and handling are much faster/easier.

Loading output library...

The NumericalSensors class is for parsing the numerical data. Lets create a dataframe for the numerical sensor data

We can access the list of sensor entities using the list_sensors attribute

Loading output library...

Now lets look at the dataframe

Loading output library...

Lets now check for correlations in the data using the all_corrs() method

Loading output library...

Unsurprisingly the mean temperature is strongly correlated with all of the temperature sensors.

Interestingly my iphone battery level is somewhat inversely correlated with the travel time from home to waterloo, which gets longer late at night when my battery level is more likely to be low.

Plot sensor data

#Plot-sensor-data

Convenience to plot a sensor data. Pass a single entity to plot:

Loading output library...

We can pass a list of entities to plot:

Loading output library...

Even mix up lists and single entites

Loading output library...

Pairplot

#Pairplot

A seaborn pair plot to show correlations.

Loading output library...
Loading output library...

Binary sensors

#Binary-sensors

The BinarySensors class is for binary sensor data with on/off states.

Loading output library...

Currently we can plot a single binary sensor with the plot() method

Loading output library...

Day of week analysis

#Day-of-week-analysis

Lets analyse the motion_at_home, create some features for day of week and time category, then analyse motion by these features.

Loading output library...
Loading output library...
Loading output library...
Loading output library...
Loading output library...