Creating a real usable phone using OsmocomBB

[Harald Welte-3]

Hi all!

I've mentioned this before, and I keep getting back to it:  With all the
great work that has been put into OsmocomBB, we are "at an arms lengh"
away from being able to create a true Free Software mobile phone.

We already have the hardware drivers, protocol stack and even the
'mobile' program which can be used for making and receiving voice calls
and sending/receiving SMS text messages on real GSM networks!

While the journey has been a lot of fun and everyone involved has
learned a lot, we have so far been catering mstly about "scratching our
own itch", i.e. implementing what we needed in order to satisfy our ego
and/or to implement the ideas we had regarding cellular security.

I believe we cannot miss the bigger opportunity here to put our code
into bigger use:  To create something like a very simple GSM feature
phone.

When we look at various areas of computing like Operating Systems or Web
browsers, Free Software is not just "the hobby project catching up" with
the vendors of proprietary software.  Free Software can compete.

In the cellular area, we have still not managed to even implement the
most basic GSM feature phone that existed 15 years ago using proprietary
software.  We need to work on closing that gap.  We need to show that a
small community of Free Software developers can actually implement what
teams of hundreds of engineers did in a proprietary software setting 15
years ago - despite all the lack of hardware documentation or any kind
of positive feedback from the cellular chipset, handset or operator
industry.

If we don't at least get a 2G GSM cellphone implemented now, it will
probably not happen before 2G networks become insignificant in large
parts of the world.

This is a call to all hands, please support this project!

Regards,
        Harald

== Technical aspects ==

I believe the first major decision is whether we focus on

1) the Openmoko FreeRunner / Neo1973 phones

Advantages:
 * large screen for UI with bells and whistles
 * lots of RAM and Flash, even script languages or compilation on the
   device itself possible
 * second processor doesn't require us to run stack + UI on once CPU
 * easier debugging of UI
 * various existing telephony middleware and phone dialer UI projects
   of which hopefully one could be recycled

or

2) the Motorola/Compal C1xx phones

Advantags:
 * many more phones available, even after our software is released
 * lower cost of the individual phone
 * less power consumption due to only one small ARM7 core
 * smaller screen also means less fancy UI requirements

Problems:
 * full stack + UI needs to run on calypso (L2/L3) and we'd probably
   some kind of RTOS like NuttX instead of our 'bare iron' code.

==== What we need in any case ====

 * power management on the baseband processor through all of the stack
   (though it's mostly a driver/L1 kind of thing)

== Summary / Opinion ==

It seems like running the OsmocomBB layer1 + 'mobile' as-is on the
Openmoko baseband + application processor might be the quicker road to
progress.  Sure, the power consumption will be horrible as the AP will
have to be woken up for each and every SI message, neighbor cell
measurment or paging request that ew see comining in in our paging group
(even in idle mode).  But then, there is always the negative impact of
using a relatively complex system, with two processors, a complex
software stack (Linux, X11, toolkit, etc.) on one of them, etc.

On the other hand, using the C1xx phones will result in a much more
widely available result.  The phones can still be bought in batches of
1,000 units, and they are small enough for lots of people to carry
around.  Furthermore, the battery lifetime is far beyond anything you
would ever be able to achieve on a power-hungry smartphone platform.  I
believe it would be the "smart' solution, as it means we need to get
everything integrated, etc.

What does the community on this list think?  Which way shoul we go?

But maybe the best thing is to actually stat working on the power
management aspects, as we will need them in both cases.

Happy hacking,
        Harald
--
- Harald Welte <[hidden email]>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

[Peter Stuge]

Harald Welte wrote:
> What does the community on this list think?  Which way shoul we go?

I think that C1xx is the only worthwhile road.

Both phones need more work to get something useful, and I think that
making it as widely available and usable as possible is highest
priority, to boost the project by also getting users involved.

Currently the threshold is high to run osmocomBB. If there is a phone
which can dial out and receive incoming calls then the threshold is
very low. Almost no UI and no features needed. Only start with the
most basic functionality: calling, and then go from there.


//Peter

[jolly]

hi,

Harald Welte wrote:

> Hi all!
>
> I've mentioned this before, and I keep getting back to it:  With all the
> great work that has been put into OsmocomBB, we are "at an arms lengh"
> away from being able to create a true Free Software mobile phone.
>
> We already have the hardware drivers, protocol stack and even the
> 'mobile' program which can be used for making and receiving voice calls
> and sending/receiving SMS text messages on real GSM networks!
>
> While the journey has been a lot of fun and everyone involved has
> learned a lot, we have so far been catering mstly about "scratching our
> own itch", i.e. implementing what we needed in order to satisfy our ego
> and/or to implement the ideas we had regarding cellular security.
>
>  

indeed it is allot of phun. the implementation of security features
(like imsi catcher detection) is not really a problem, but therefore we
must define them first.
> I believe we cannot miss the bigger opportunity here to put our code
> into bigger use:  To create something like a very simple GSM feature
> phone.
>
>  
we should not only try to implement a 10 euro phone. this is what we
already had before the osmocombb project. osmocombb should focus on
things that normal phones doesn't have. but yes, it must first do the
basic stuff correctly, like being stable and usable like a GSM phone.
otherwise it is a pain using it. i think of two issues still left for
the stack: reliable syncing and handover.

> When we look at various areas of computing like Operating Systems or Web
> browsers, Free Software is not just "the hobby project catching up" with
> the vendors of proprietary software.  Free Software can compete.
>
> In the cellular area, we have still not managed to even implement the
> most basic GSM feature phone that existed 15 years ago using proprietary
> software.  We need to work on closing that gap.  We need to show that a
> small community of Free Software developers can actually implement what
> teams of hundreds of engineers did in a proprietary software setting 15
> years ago - despite all the lack of hardware documentation or any kind
> of positive feedback from the cellular chipset, handset or operator
> industry.
>
> If we don't at least get a 2G GSM cellphone implemented now, it will
> probably not happen before 2G networks become insignificant in large
> parts of the world.
>
> This is a call to all hands, please support this project!
>
> Regards,
> Harald
>
> == Technical aspects ==
>
> I believe the first major decision is whether we focus on
>
> 1) the Openmoko FreeRunner / Neo1973 phones
>
> Advantages:
>  * large screen for UI with bells and whistles
>  * lots of RAM and Flash, even script languages or compilation on the
>    device itself possible
>  * second processor doesn't require us to run stack + UI on once CPU
>  * easier debugging of UI
>  * various existing telephony middleware and phone dialer UI projects
>    of which hopefully one could be recycled
>
> or
>
> 2) the Motorola/Compal C1xx phones
>
> Advantags:
>  * many more phones available, even after our software is released
>  * lower cost of the individual phone
>  * less power consumption due to only one small ARM7 core
>  * smaller screen also means less fancy UI requirements
>
> Problems:
>  * full stack + UI needs to run on calypso (L2/L3) and we'd probably
>    some kind of RTOS like NuttX instead of our 'bare iron' code.
>
>  

i prefer this one, because it will have (and already has) a much larger
audience. if we would have the RTOS working and finished the UI (as well
as defining an MMI interface for communication with the stack), we
should have almost made it.

> ==== What we need in any case ====
>
>  * power management on the baseband processor through all of the stack
>    (though it's mostly a driver/L1 kind of thing)
>
> == Summary / Opinion ==
>
> It seems like running the OsmocomBB layer1 + 'mobile' as-is on the
> Openmoko baseband + application processor might be the quicker road to
> progress.  Sure, the power consumption will be horrible as the AP will
> have to be woken up for each and every SI message, neighbor cell
> measurment or paging request that ew see comining in in our paging group
> (even in idle mode).  But then, there is always the negative impact of
> using a relatively complex system, with two processors, a complex
> software stack (Linux, X11, toolkit, etc.) on one of them, etc.
>
> On the other hand, using the C1xx phones will result in a much more
> widely available result.  The phones can still be bought in batches of
> 1,000 units, and they are small enough for lots of people to carry
> around.  Furthermore, the battery lifetime is far beyond anything you
> would ever be able to achieve on a power-hungry smartphone platform.  I
> believe it would be the "smart' solution, as it means we need to get
> everything integrated, etc.
>
> What does the community on this list think?  Which way shoul we go?
>
> But maybe the best thing is to actually stat working on the power
> management aspects, as we will need them in both cases.
>
> Happy hacking,
> Harald
>  

we should define all that at 28c3, so we can start to complete it.

regards,

andreas

[Scott Weisman]

I also vote for the C1xx phones. I didn't realize they were still available in such large quantities, which itself is a huge advantage IMHO. But the simpler development model and needed software stack, along with major power advantages seems too good to ignore.

I don't know if the previous discussion/decision regarding NuttX stands. This (limited) RTOS benchmark looked interesting to me:

http://www.yagarto.de/projects/rtoscomp/index.html

I am very interested in contributing what I can to such a project.

Scott

[Harald Welte-3]

Hi Scott,

On Mon, Dec 12, 2011 at 12:17:35PM +0200, Scott Weisman wrote:

> I don't know if the previous discussion/decision regarding NuttX stands.
> This (limited) RTOS benchmark looked interesting to me:
>
> http://www.yagarto.de/projects/rtoscomp/index.html

I am not overly worried by that.  From my point of view, aspects like
the POSIX-style API and the general structure of the code have much more
significance than absolute scheduler latency.

It would be great to see some work on getting more of our existing
drivers for the calypso ported into NuttX.

Regards,
        Harald
--
- Harald Welte <[hidden email]>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

[Alan Carvalho de Assis]

Hi Scott,

On 12/12/11, Scott Weisman <[hidden email]> wrote:

> I also vote for the C1xx phones. I didn't realize they were still available
> in such large quantities, which itself is a huge advantage IMHO. But the
> simpler development model and needed software stack, along with major power
> advantages seems too good to ignore.
>
> I don't know if the previous discussion/decision regarding NuttX stands.
> This (limited) RTOS benchmark looked interesting to me:
>
> http://www.yagarto.de/projects/rtoscomp/index.html
>

Please don't confuse Nut/OS with NuttX, they are different RTOS.
NuttX is a kind of POSIX/"Unix-like" for micro-controller.

> I am very interested in contributing what I can to such a project.
>

You can start looking the NuttX port for Calypso micro-controller
which Stefan ([hidden email]) developed.

Best Regards,

Alan

[Michael Sokolov]

Hello folks,

Just my two cents: one factor that seems to be overlooked in the C1xx
vs. GTA0x decision is the support for different GSM RF bands.  From my
reading of the OsmocomBB wiki, it appears that all those C1xx phones
support only EGSM and DCS1800 bands, but not PCS1900.  It seems like
most current active developers are located in world regions where EGSM
and DCS1800 bands are used, but in my area the only usable GSM band is
PCS1900, or maybe GSM850 too: haven't been able to try the latter as I
don't have any devices that support it.

OTOH, my Neo Freerunner (which is currently doing shelf duty as a
paperweight lacking functional source-enabled Calypso fw) supports the
PCS1900 band in addition to EGSM and DCS1800.

But if the community goes the C1xx route and produces GPL code that
runs fully on the Calypso, no PC needed, does power mgmt and
implements some UI on the Calypso-controlled LCD and keypad, I expect
no difficulty with taking that code, replacing the Calypso-based UI
with some simple protocol for interfacing with the UI over the modem
UART (maybe not the same as the infamous AT command interface, but
doing the same job on the same level of abstraction), and running it
on my GTA02, hence I'm not complaining.

As far as the power management etc advantages of the simpler phones
without an application processor: does anyone know of any basic
Leonardo-style phones which are like the famous C1xx, but support the
PCS1900 band, are physically available, and documented no worse than
C1xx in terms of the availability of schematics and docs for display
and other non-Calypso components?

The fact that C1xx phones can be bought in 1000 unit quantities is
great news for those living in areas with EGSM/DCS1800 services, but
doesn't do much good for those living in PCS1900/GSM850 lands...

MS

[Lokkju Brennr]

I'm running a C117 on PCS1900 with no problems, in the USA.  It was an
old Tracfone I fixed the bootloader on and re-flashed.  I have no idea
if PCS1900 type C117 phones are widely available - but I do know they
exist.  Interestingly, I recently saw one in the local Walgreens for
$20, brand new.

Loki

On Mon, Dec 12, 2011 at 9:12 AM, Michael Sokolov
<[hidden email]> wrote:

> Hello folks,
>
> Just my two cents: one factor that seems to be overlooked in the C1xx
> vs. GTA0x decision is the support for different GSM RF bands.  From my
> reading of the OsmocomBB wiki, it appears that all those C1xx phones
> support only EGSM and DCS1800 bands, but not PCS1900.  It seems like
> most current active developers are located in world regions where EGSM
> and DCS1800 bands are used, but in my area the only usable GSM band is
> PCS1900, or maybe GSM850 too: haven't been able to try the latter as I
> don't have any devices that support it.
>
> OTOH, my Neo Freerunner (which is currently doing shelf duty as a
> paperweight lacking functional source-enabled Calypso fw) supports the
> PCS1900 band in addition to EGSM and DCS1800.
>
> But if the community goes the C1xx route and produces GPL code that
> runs fully on the Calypso, no PC needed, does power mgmt and
> implements some UI on the Calypso-controlled LCD and keypad, I expect
> no difficulty with taking that code, replacing the Calypso-based UI
> with some simple protocol for interfacing with the UI over the modem
> UART (maybe not the same as the infamous AT command interface, but
> doing the same job on the same level of abstraction), and running it
> on my GTA02, hence I'm not complaining.
>
> As far as the power management etc advantages of the simpler phones
> without an application processor: does anyone know of any basic
> Leonardo-style phones which are like the famous C1xx, but support the
> PCS1900 band, are physically available, and documented no worse than
> C1xx in terms of the availability of schematics and docs for display
> and other non-Calypso components?
>
> The fact that C1xx phones can be bought in 1000 unit quantities is
> great news for those living in areas with EGSM/DCS1800 services, but
> doesn't do much good for those living in PCS1900/GSM850 lands...
>
> MS
>

[Denis 'GNUtoo' Carikli]

>But if the community goes the C1xx route and produces GPL code that
>runs fully on the Calypso, no PC needed, does power mgmt and
>implements some UI on the Calypso-controlled LCD and keypad, I expect
>no difficulty with taking that code, replacing the Calypso-based UI
>with some simple protocol for interfacing with the UI over the modem
>UART (maybe not the same as the infamous AT command interface, but
>doing the same job on the same level of abstraction), and running it
>on my GTA02, hence I'm not complaining.
That's exactly what I was thinking about, since running the layer23 on the
Application CPU is not a so good idea, it renders the phone usable only for a
small period of time(since,in that configuration, you cannot suspend the AP
CPU because it has to run the telephony code).

>maybe not the same as the infamous AT command interface
Why is AT bad? what other protocol do you propose?

Denis.

[Peter Stuge]

Denis 'GNUtoo' Carikli wrote:
> Why is AT bad?

Um, no, it's the other way around. AT is 30 years of horrible.


//Peter

[Michael Sokolov]

Denis 'GNUtoo' Carikli <[hidden email]> wrote:

> That's exactly what I was thinking about, since running the layer23 on the
> Application CPU is not a so good idea, it renders the phone usable only for a
> small period of time(since,in that configuration, you cannot suspend the AP
> CPU because it has to run the telephony code).

It's obvious that running layer23 on the AP is a *terrible* idea.  It
needs to move into the Calypso (or other BP) before there can be any
serious thought of OsmocomBB truly replacing the existing proprietary
BP code.  However, my own skill & knowledge level is nowhere close to
what would be needed for that job.

But I was thinking of taking the current sorry state of things
(layer23 external to the BP) and making it run on a self-contained
GTA02: have the Samsung AP run layer23 instead of acting as a pass-
through to a PC, then add some UI to it: the very same UI I wanted to
implement in the first place before I got sidetracked to the baseband
issues.  The result of that would be having the GTA02 act as a
"normal" feature phone in every way except for draining the battery in
a couple of hours while doing nothing.  The last aspect would make it
rather unusable practically (unless perhaps one replaces the standard
Li-ion battery with an automotive-sized lead-acid one), but it would
be a tangible hands-on "teaser" of what would be possible if someone
with the necessary knowledge and skills took the bother to redesign
the code (move layer23 into the Calypso) such that proper PM could be
implemented.

> Why is AT bad? what other protocol do you propose?

What Peter said:

: Um, no, it's the other way around. AT is 30 years of horrible.

If I were able to lay my hands on some source for an *existing*
implementation of the AT command interface on the BP side, I would
have no problem with implementing it on the AP side in my UI.  But if
I have to write the code from scratch on both sides of the interface,
I'll probably implement some ad hoc protocol of my own rather than
attempt to implement the "standard" one and get it wrong.

MS

[Harald Welte-3]

On Mon, Dec 12, 2011 at 07:02:21PM +0100, Denis 'GNUtoo' Carikli wrote:

> >maybe not the same as the infamous AT command interface
>
> Why is AT bad? what other protocol do you propose?

I've you've ever tried to write any fully-fledged AT command parser
(like the various incarnations of Openmoko gsmd/libgsm, fso gsmd, ofono,
etc.) then you know why.  It's nice for human beings, but it's horribly
overloaded for any machine based parsing, especially if you only have
one channels and need to deal with unsolicited results overlapping with
synchronous request/response type commands at the same time.

In any interface where you have asynchronous signalling, each command
should be tagged with an identifier, which is contained in the
corresponding response.  This is done e.g. very nicely in IMAP and you
can have as many outstanding/executing commands in parallel as you want,
without any difficulty parsing the responses whatsoever.

Regards,
        Harald

--
- Harald Welte <[hidden email]>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

[Jay R. Worthington]

Hi Harald,

2011/12/11 Harald Welte <[hidden email]>

If we don't at least get a 2G GSM cellphone implemented now, it will
probably not happen before 2G networks become insignificant in large
parts of the world.

what do you expect to happen from a legal standpoint? My guess would be that the providers will fight an opensource firmware with every firebreathing lawyer in der reach, and if they won't do that, RegTP (or whatever they are call themself this week ;)) for sure will, a firmware that would reject some evil RRLP queries can't be tolerated :-S

[Peter Stuge]

Jay R. Worthington wrote:
> what do you expect to happen from a legal standpoint? My guess
> would be that the providers will fight an opensource firmware with
> every firebreathing lawyer in der reach,

Translate your question to wifi.

Operators can of course try even harder to control in agreements what
device you are allowed to use their SIM in, but I don't think that
will hold for very long.


//Peter

[Paul Wise]

As a gta02 owner, I am biased towards that.

As a lurker and osmocom supporter, the C1xx phones seem to have more
of a future.

I also wonder which work will also help for future ports such as MTK.

The main issue with creating a usable phone though is getting people
to use it. For gta02 you can simply get osmocombb included in SHR,
QtMoko, Debian etc and voilà, instant users who care about software
freedom. For C1xx you would need to create a user community from
scratch and do a lot of work to reach the people who might be
interested.

In both cases there is the whole legality/certification hurdle though.

--
bye,
pabs

http://bonedaddy.net/pabs3/

[suraev]

13.12.2011 02:02, Denis 'GNUtoo' Carikli пишет:

> That's exactly what I was thinking about, since running the layer23 on the
> Application CPU is not a so good idea, it renders the phone usable only for a
> small period of time(since,in that configuration, you cannot suspend the AP
> CPU because it has to run the telephony code).

I've probably missed this part of discussion. Could you re-iterate: is it technical
limitations that prevent us from porting layer23 onto calypso BP or it's simply lack
of manpower?

bye,
Max.

[Harald Welte-3]

On Tue, Dec 13, 2011 at 12:56:08PM +0800, [hidden email] wrote:

> 13.12.2011 02:02, Denis 'GNUtoo' Carikli пишет:
>
> > That's exactly what I was thinking about, since running the layer23 on the
> > Application CPU is not a so good idea, it renders the phone usable only for a
> > small period of time(since,in that configuration, you cannot suspend the AP
> > CPU because it has to run the telephony code).
>
> I've probably missed this part of discussion. Could you re-iterate: is it technical
> limitations that prevent us from porting layer23 onto calypso BP or it's simply lack
> of manpower?

It's really only the latter.  There are probably a number of external
references that need to be resolved (we don't have a full libc/libm in
the target), as well as issues related to unaligned accesses.

Also, the current 'mobile' code probably needs to be split into the
actual 'logic' code (L2, L3, cell [re]selection, etc.) and the
'application' part (the VTY based interface we have right now).  The
higher-level interface of the 'logic' code and the application needs to
be defined, probably as some kind of message queue of primitives.

A first incremental step could be to move L2 (LAPDm) into the phone and
use RSLms to talk with the existing PC-based 'mobile' application over
the UART.

Regards,
        Harald.
--
- Harald Welte <[hidden email]>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

[Harald Welte-3]

On Tue, Dec 13, 2011 at 12:59:08AM +0100, Jay R. Worthington wrote:

> what do you expect to happen from a legal standpoint? My guess would be
> that the providers will fight an opensource firmware with every
> firebreathing lawyer in der reach, and if they won't do that, RegTP (or
> whatever they are call themself this week ;)) for sure will, a firmware
> that would reject some evil RRLP queries can't be tolerated :-S

Hi Jay,

in fact, my legal analysis had been quite optimistic, at least for
Europe.  the RT&TTE directive largely regulates the sale and
distribution of "devices" that transmit on radio frequencies.  Devices
need to have CE markings and a declaration of conformity.  As GSM
terminals are part of harmonized standards, the vendor can either
certify himself that the devices are CE compliant, or he can use a
'notified body' (a certification lab) to do that externally.  The
Procedure is described in Annex III of the directive.

The testing that needs to be done is in EN 301 511, and EN 301 489-7

However, this all only applies if you distribute the devices with
modified firmware.  The device with original firmware of course is
compliant to the directive and has a Motorola declaration of conformity.

Distributing the OsmocomBB firmware itself is certainly not a "device"
under the current legislation.

Installing + Using it as a user [on a public network] might pose a legal
risk, but to be honest I wouldn't know what kind of regulation that
would be.   There might be a breach of contract of your operator terms
of services.  And of course, if the firmware misbehaves and causes RF
interference, that would be transmission without a radio license, or in
the worst case interference with public communications networks.

But then, at the same time, lots of people already use Free Software
based firmware in their WiFi chips, and I think we've had a lot of
discussion in that area.  Nonetheless, many people do it...

An no, there is no real difference here due to the fact that 2.4 GHz ist
unregulated spectrum.  You also have to make sure that the frequency,
transmission power, harmonics, etc. fall within the rules set forth in
the harmonized standards.

Regards,
        Harald
--
- Harald Welte <[hidden email]>           http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

[Peter Stuge]

Harald Welte wrote:
> There are probably a number of external references that need to be
> resolved (we don't have a full libc/libm in the target),

Look at libpayload. It's a libc-like project for writing coreboot
payloads, ie. running on bare metal, and implements some of libc
either on it's own or by borrowing from other OS projects. BSD
license.


> Also, the current 'mobile' code probably needs to be split into the
> actual 'logic' code (L2, L3, cell [re]selection, etc.) and the
> 'application' part (the VTY based interface we have right now).

I think this is a great first step, and a very easy way to get
involved for someone who is not yet so experienced with GSM, but
already has a good understanding of C on PCs.


//Peter

[jolly]

Peter Stuge wrote:
>> Also, the current 'mobile' code probably needs to be split into the
>> > actual 'logic' code (L2, L3, cell [re]selection, etc.) and the
>> > 'application' part (the VTY based interface we have right now).
>>    
> I think this is a great first step, and a very easy way to get
> involved for someone who is not yet so experienced with GSM, but
> already has a good understanding of C on PCs.
>  
i think it would help if we look at the VTY commands and what they tell
the stack and what they receive as response. i suggest defining "MMI_*"
message primitves for an interface. other applications, like the UI
could use and connect to the same interface. (possibly other
applications like AT command interface) i suggest defining the if at 28c3.