Open Source Mobile Communication

I’ve been thinking about this for a year now. The idea was probably spawned from T-Mobile providing a wireless router that you could use your phone to make calls through and then given more life with the emergence of everyone else providing femtocells.

What’s the idea? Well, it goes something like this: Why am I paying $400 for a computer I have limited access to and $100+ to have access to a network that provides limited functionality?

First, is such a thing possible? Well, it’s been done before? Yes:

https://www.engadget.com/2012/04/25/mit-media-lab-diy-cellphone-hands-on/

But I’m not an open source zealot - if it isn’t better than a closed source alternative, I won’t buy it (or invest time into it). What MIT did was awesome. But this type of device is sub-par to me. Endgadget said it won’t run Android. That’s fine with me - it could run a Nokia Linux or Windows or IOS or another platform and that would be okay with me. However, what they meant is that it has a limited operating system such as a phone 10+ years ago might have - no web browser, email, or other mobile applications we’ve become so fond of. This simply won’t do.

What’s my motivation? Knowing that my hardware and the Android OS I was using had limitations is certainly annoying but it isn’t motivating. Paying $130 per month for unlimited everything, a 4G connection, insurance, tax, etc is definitely motivating as well. However, this idea was still just an itch that didn’t go away and I didn’t really scratch much - I used Skype and Google Voice but it was more of a consideration of whether my cell phone or computer was closer than anything else. Then, a few days ago something happened - I lost my cell phone. I lost it in a taxi, or maybe on a sidewalk outside the cab - who knows, no one has found a black Sprint branded HTC Evo 4G. Today, I went by a Sprint store to confirm my options: I can buy another cell phone, I can buy another subsidized cell phone with a new contract, or I can cancel my service.

Decisions. I pretty much knew what I was going to have to do before I walked into the store. Even though I use my Sprint phone number directly with Google Voice, if I cancel my service, I loose my phone number (it doesn’t stay with my Google account). I am not going to pay $50 for a 2+ year old cell phone (unless a new refurbished Evo will sell for more on craigslist - I’ve got a few days to consider this). And I don’t want to pay $40+ for a service I have little use for (mobile voice communication). So I lowered my plan to the lowest smart phone plan they have for my 4G phone - $80 per month. Now I need to transfer the phone number to a VoIP provider and cancel the plan.

This is all well and good if I only want a land line type phone service, however I really do want more. So what then? A few months ago, I got this cool little computer called a Raspberry Pi. I’ve used it for some things - contributed some packages to ArchLinux-ARM mainly, but not much more. However, what if I could make calls on my little credit card sized PC?

What are the hurdles to using the Raspberry as a wired telephone? The audio receptacle only has 3 contacts and is only used for output (speaker) - no input or mic functionality. This is a hardware limitation that I don’t know how to get past. However, this isn’t a deal breaker. I could get a USB sound card and plug in a mic / headphone unit into that. But why? That costs $50+, adds bulk, and teathers me to the unit. Instead, I’ve got this little Kingston USB Bluetooth device that isn’t being used for anything and is so small I’m surprised I haven’t lost it yet (it cost me about $40 at a brick and molder store 3 years ago). I used to have a Jawbone Icon Bluetooth ear peace (those things that allow you to look like you’re talking to yourself in public) and I really liked it. I found out that I could get the next version up, the Jawbone Icon HD for $50 with shipping.

So, I’ll be able to talk on a phone system now. However, I’m still semi tethered - the Raspberry Pi only has an Ethernet port (no WiFi), there is no battery, and no display. This simply won’t do for a mobile device. The first thing to fix is the WiFi which is the easiest and cheapest. The Asus USB-N10 is a USB WiFi adapter with good coverage, a decent feature set, is almost as small as my Bluetooth adapter and can be had for $20 from MicroCenter. Even with Bluetooth and WiFi, I’ve still got power to contend with. I should have a plug ‘n play solution for this with these two components:

https://www.sparkfun.com/products/11231

https://www.sparkfun.com/products/8484

For $60 I should be able to power the Raspberry Pi for 4+ hours. The only issue I’m going to need to resolve is getting a battery meter. This might be a bit tricky because of the circuitry in the battery pack (but doable).

At this point, I’ll need to do some actual mechanical engineering in order to keep things from flopping around, chips from getting dislodged and debris from shorting anything. I haven’t really done any 3d printing before, but I’ve done a little with AutoCAD and am hoping this won’t be too much of an issue. I’m also considering removing most of the GPIO header pins on the Raspberry Pi me to make this more compact (or maybe put a buffer material here). There is also the matter of the micro-USB power adapter pointing straight out which should be fixed in order to conserve space. Lastly, something is going to have to be done with the SD card - it’s huge and worthless since I just plug an adapter into it. I’m hoping I can either find a half height adapter or an adapter board that has solder points for the larger interface but with the slot for the smaller micro SD card.

Cell phones have this cool technology that allows your signal to jump from one tower to another and you maintain your call or network connection (generally). Wireless technologies also allow this but you have to manage the entire infrastructure you hope your signal to hop on and off of. If I ever intend to walk out of the house without the need to re-establish a call, I will need to be able to hop from one network to another and re-establish my telephone connection. In normal circumstances, your VoIP call will hang up if you loose connection. Though, I don’t want to ‘loose’ connection, that’s essentially what will happen every time I hop from one network to another.- VCC is the VoIP solution for this but my understanding is that it needs to be implemented by every network you hop to.

The ideal setup would be to have another small USB device that transfers the call to a cellular network whenever VoIP traffic degrades too much or WiFi signal strength / bandwidth drops to a certain level. This isn’t possible for two reasons: first, because the Raspberry Pi only has two USB ports - though this could be resolved with a hub, this adds more bulk and there are power constraints to consider as well. The second reason is even more severe from where I stand: most mobile USB cards don’t have drivers for Linux, are just as expensive as the alternative (WiFi hotspot), can break easily in a setup like this. Maybe someone will develop a cellular modem that plugs into GPIO on the Pi but otherwise I’ll have to carry around another device if I go somewhere without WiFi.

In order to jump between networks and maintain a VoIP connection, I was thinking that MoSH would be perfect (https://mosh.mit.edu). Alas, they haven’t implemented a way to do forwarding (of any kind it seems) like what you can do with SSH. The alternative is using OpenVPN with the server allowing a remote peer to change their ip (–float) and with the client always having a fail over connection. The issue with this will be figuring out when to fail over (and when not to) since you can only have one connection open at a time (maybe a fail over user just for figuring out the alternative connection speed / latency?). I don’t know too much of how this part is going to work as I just figured out that MoSH wasn’t going to work.

Lastly there’s the issue of input and output while being mobile. There are three ways of dealing with input: keyboard, a touchscreen, and voice. There are two ways of dealing with output: a screen, and through speech synthesis. Dealing with output is the easiest to figure out - you already have an audio device in your ear, so you might as well use it. However, sometimes pictures are better than words. In the later situation, the only question to ask is where to put the screen? If I’m sitting on the couch, in a car or even at a bar, a pico projector is an obvious choice. But what if I’m walking? A screen built into the device works 99% of the time for most people - people run into other people and things and looking down at a screen interrupts conversations, but this generally works. But, what if I’m running, biking, or driving? Looking down at a device doesn’t work and reading at the edge of your periphery is hard and annoying. For this, there are heads up displays (HUDs). Black hawk helicopters have had this type of unit for decades. And we can buy them too - they’re $300~600 and are meant for your living room or an airplane flight where you might want to watch a movie and nothing else. If i try to wear one of these units and ride a bike, really bad things are likely to happen (well, probably just somewhat bad things since I’m probably going to hit something before I get going too fast). There is a company that makes a HUD that only blocks your vision with the content it needs to display - so like the HUD in a BMW, you can see around the numbers of your odometer and such. They only sell in bulk to whole sellers so I have no idea how much their products cost: https://www.lumusvision.com/

However, this looks like the perfect application for my use case. There was also another product that looks like it might have worked: Microoptical SV-6 (though I don’t think the company is in business anymore and the product looked a bit bulky).

So, besides audio synthesis (which shouldn’t be too hard and I’ll probably use Festival for) and a pico projector, we’re left with an integrated display for user output. There are inexpensive displays to be had but nothing too exciting. Just for an example:

https://www.alibaba.com/product-gs/553291029/2_4_inch_resitive_lcd_screen.html

https://www.digikey.com/product-detail/en/NHD-3.12-25664UCY2/NHD-3.12-25664UCY2-ND/2640786?cur=USD

Neither are touch and the later is $30 for a monochrome unit that would require some effort in order to get it to work. The former might be more promising - it’s smaller, costs $3~8, and takes HDIM video input. What would be required for this to become useful for me is to figure out how to stitch the two or more displays together without additional bulky hardware or a big price tag.

Lastly, there’s input. This is somewhat reliant on what my output is and on the environment. Ie, you can’t use a pointing device without a screen, i’m not going to pull out a QWERTY keyboard while walking down a sidewalk (I’ve tried this with a laptop - it doesn’t work for long distances), and I’m not going to speak to a screen (sorry doctors, but it looks totally stupid and I can type about as fast as I want to convey ideas in this medium). However, given that I’ve already got audio going, I’m going to want speech recognition first. Julius seems to have the happy medium of being the best, not so hard to setup, open source software. Besides, someone has already written a tutorial showing how to get it setup on the Raspberry Pi: https://www.aonsquared.co.uk/raspi_voice_control

What else is required for our voice input? It would be nice to figure out how to see how long the Jawbone’s function button had been pressed so that you could have different options for input (maybe send the audio to a remote computer with more power in noisy environments with a longer press).

Though you wouldn’t want to break out a full keyboard while mobile, current mobile input mechanisms are clumsy and slow. Namely the T9 keyboard that older phones had, the BlackBerry miniature QWERTY keyboard, and the soft keyboards on most phones today. Some are better than others because of size or tactual nature. But they’re still not as good to type on as a full sized keyboard. An alternative might be the Bat keyboard - a one handed keyboard that should allow one to type almost as fast as a full sized keyboard. I know someone who is working on the hardware to make this look like a normal USB HID (human interface device). If he succeeds, I’d like to take the electronics and put them in a semi-rigid glove so that you close your hand enough, a hinge or other system stops your finger from moving farther so that the key is pressed. This would allow you to have general use of your hand but still type.

Sometimes gesture controls are nice to have however. So, it might be nice to integrate a touch screen (instead of a normal LCD). It would also be nice to mount a stereo camera setup (or other outward tracking system) on LumusVision glasses. But since a device mounted display isn’t optimal and I can’t purchase LumusVision glasses, I’m not really pursuing either.

I’ve seen innovations with wearable computers over the past decade or so. These setups still look strange. If this setup is to succeed, I will not have wires running all around me - that’s just not fun, hazard prone, and looks stupid. This will also need a decent battery life and charge time. I don’t want something too long - I never remember to charge an iPad until it has 10% battery and I’m not around an outlet. The power setup should last long enough for me to get between charging stations and be a nominal task to plug in - if I need to plug in a half dozen batteries, this has failed. I need to be able to easily swap out batteries - if I need to be out longer than a set of batteries last, I don’t want do be left without communications. I want to be able to change out any component on the fly and have the system work with new components with minimal setup - it shouldn’t care whether I’ve got a Bluetooth Bat keyboard glove or a touch screen connected for input and should process them the same way and if my Jawbone dies I shouldn’t be disconnected (maybe I can transfer the call).

What do I plan to do next? Transfer my phone number to a VoIP service and get a prepaid phone. Then get my VoIP number to ring my Google Voice number and the number of the prepaid phone. Then work out all of the quirks with the Jawbone and my Bluetooth radio on the Pi with Pulseaudio, Bluez, Festival, Julius. After that, I’ll want to look into the battery, wireless card, OpenVPN, and a WiFi hotspot. I suspect, by the time I get this part figured out, I might be reevaluating some the choices for my hardware and software stack. We’ll see where this goes but needless to say, I’m not going to pay $130 for mobile connectivity anymore.