(I'm moving all my posts from Blogger to this site. You may see some missing links in the process.)
As part of our holiday cleanup, I spent some time optimizing the Wifi layout of our house. To help with this, I built an AR app "AR Sensor" to measure and visualize various spatial data, such as the Wifi signal strength:
The above gif shows what the output of AR Sensor looks like in our house; it shows how the Wifi signal strength received by the smartphone gets better (goes greener) as I go up the stairs (because the router is on the upper floor), and all of the historical measurements are "recorded" in the space.
The way it works is very simple. It uses the positional tracker provided by ARCore (Android's default AR framework) to constantly track the 3D coordinates of the phone inside the house, whilst also measuring other data provided by the smartphone's sensors (like Wifi signal strength). As you move the phone about, it accumulates the measurements which you can view in AR:
This lets us "see" sensor data in its spatial context, letting us better understand its 3D characteristics and distribution in the environment.
What else can it do?
I've added support for 5 different sensors in the first release of AR Sensor: "Wifi Signal Strength", "Bluetooth LE Signal Strength", "Magnetic Field", "Barometric Pressure", and "LTE (Mobile Cell) signal strength". Let me explain how each of these are interesting.
Wifi Signal Strength
Here's a fuller version of the gif posted above. It helps visualize how a flight of stairs or even a single wall is enough to bring the signal (visually) down. I've since moved the Wifi router to the top of the staircase, which let it cover the entire house.
ガリのほう） (@kenkawakenkenke) December 24, 2018
Bluetooth LE Signal Strength
ガリのほう） (@kenkawakenkenke) December 30, 2018
Using the magnetometer, we can measure the direction and strength of the magnetic field at points in space. Inside a typical room this is fairly unappealing; the particles drift towards North slightly towards the ground (i.e., towards the magnetic north pole):
but when placed near a magnet, it can create a fantastic visualization of the magnetic field around the magnet:
ガリのほう） (@kenkawakenkenke) December 29, 2018
or around two magnets, the field lines going between the magnets:
２つの磁石の間を磁力線がブワー pic.twitter.com/2JRUFKtIzc— Ken Kawamoto（ガリのほう） (@kenkawakenkenke) December 30, 2018
Not very interesting, it's basically just the height. Let me skip.
LTE Cell Signal Strength
ガリのほう） (@kenkawakenkenke) January 6, 2019
It's a pretty interesting experience walking around the city visualizing little spots of bad connection (e.g., under playground slides).
I setup a simple gas sensor TGS2600, to measure the gas density while boiling water on the gas stove. See how the gas density increases as we go near the ceiling:
Think about all the other sensors we could attach to the phone, like humidity sensors, CO2 sensors, pollen/dust sensors, etc! I'd love to hear about ideas for things to measure in the comments.
For about 10 years I've wanted to create a tool for measuring things and plotting them in 3D, but never really had the time to do implement the positional tracking alternatives that existed at the time, such as placing QR codes all around the house, tracking myself with a Kinect, and writing my own SLAM. In writing this app, I was astounded by how I could just rely on ARCore (and probably the same for ARKit) so that the end-to-end process of positional tracking, sensing, and visualization can all fit into one smartphone, with very minimal code. If you've had ideas that rely on positional tracking, now's probably a great time to implement it!