BREW/Mobile phone coding

[Nev]

C and C++ can indeed be low level. But in C++, through the STL and the abstraction mechanisms provided by the language, it need not be.

I *wish* I had STL support in writing for mobile devices. Sadly, it's not to be. I actually wrote my own abstract "container" class with iterator functionality because BREW has nothing like that. It's like stepping back in time ten years...

[Kupek]

I *wish* I had STL support in writing for mobile devices. Sadly, it's not to be. I actually wrote my own abstract "container" class with iterator functionality because BREW has nothing like that. It's like stepping back in time ten years...

Huh. I take it BREW is a development framework for embedded devices? What at the reasons for not being able to us the STL?

[Nev]

A couple things: 1. Code size easily starts to get prohibitive when linking in external libraries - even the C/C++ standard library implementations add too much - since even the biggest games are still trying to be kept below, say, 400K. 2. They aren't Intel/IBM processors (I think, I'm not really too great on the hardware end) - they're ARM-compatible - so any low-level routines that are dependent on a specific processor I think won't work and have to be rewritten.

The BREW framework overrides most of the commonly used standard library routines (malloc, free, and the string stuff), but I think there may even be some stdlib calls that aren't supported, and as for more advanced stuff like the STL, forget about it. The ARM compiler has pretty good basic syntax support, though. I think the only widely supported C/C++ feature it doesn't do right now is namespaces.

[Kupek]

A couple things: 1. Code size easily starts to get prohibitive when linking in external libraries - even the C/C++ standard library implementations add too much - since even the biggest games are still trying to be kept below, say, 400K.

Hmm. I'm not sure if that makes sense. If you need a list, then you need a list. So either you can use a standard one, or you'll have to roll your own. Either way, that list is going to take up space in the binary, it's unavoidable. But a standard data structure is probably going to be faster, smaller, and more likely correct than anything you implement yourself. (STL data structures - when used correctly - consistently outperform anything I write.)

2. They aren't Intel/IBM processors (I think, I'm not really too great on the hardware end) - they're ARM-compatible - so any low-level routines that are dependent on a specific processor I think won't work and have to be rewritten.

The STL is written entirely in C++, so portability isn't an issue. I'm not sure about the C standard libraries, but I think they're entirely in C. Regardless, gcc is supported on ARM, so ARM compatible libraries exist, but they might not be compatible with BREW. (I'm still unsure exactly what BREW is, I'm mostly assuming here from things you've said.)

The BREW framework overrides most of the commonly used standard library routines (malloc, free, and the string stuff), but I think there may even be some stdlib calls that aren't supported, and as for more advanced stuff like the STL, forget about it. The ARM compiler has pretty good basic syntax support, though. I think the only widely supported C/C++ feature it doesn't do right now is namespaces.

That's probably the reason right there. If the compiler can't handle namespaces, then you can't use the STL. (Unless, of course, you manually went in and removed all of the <TT>namespace std</TT> lines, which would be a pain in the ass.) If the BREW framework uses its own compiler, it might not (yet) be able to handle techniques used in most STL implementations.

[Ishamael]

FYI, BREW is a framework for developing mobile phone software. There's a bunch out there including WindowMobile from MS, some of the J2ME stuff Sun, etc.

On Mental's code size comment. Mobile phone used to be really bad about this (I worked for a company that developed software for mobile phones for a while). Phone carriers/manufacturers have tight control over virtually every byte of space on the device. It used to be that some manufacturers/carriers were not willing allocate the space needed for something like the STL to be stored, so you had to roll your own abstract data types every time. (Realistically of course, I'd imagine most developers took an existing one and tweaked it so that it worked within the constraints provided, rather than roll from scratch.) Also, with the limited memory, you had to be very careful about implementations as they could be making assumptions that are reasonable for a desktop, but are out of the question for an embedded device. It's pretty amazing how much the world changes at that level. However, it's getting better every year and many of the newer mobile phones are extremely powerful, thus allowing some of the constraints to be loosened.

I'm only familiar with BREW at an ARM's length distance. I see them listed in many a powerpoint, and I've heard them mentioned in many a company meeting, but I don't know much about what they offer on a technical level. I'd imagine that a lot of the details of mobile development are hidden by the BREW abstraction.

[Kupek]

On Mental's code size comment. Mobile phone used to be really bad about this (I worked for a company that developed software for mobile phones for a while). Phone carriers/manufacturers have tight control over virtually every byte of space on the device. It used to be that some manufacturers/carriers were not willing allocate the space needed for something like the STL to be stored, so you had to roll your own abstract data types every time. (Realistically of course, I'd imagine most developers took an existing one and tweaked it so that it worked within the constraints provided, rather than roll from scratch.)

I still don't follow the reasoning, but I think I know why. The STL isn't a library that you link to and gets called at run time. When you use the STL, it gets compiled in with your binary. It has to work this way because of how templates work in C++. (The definition is required at compile time; you can't just provide the interface.) So there's really no space overhead in using it, because one way or another, that data structure is going to take up space. Programs that don't use it won't have to pay for it, either.

Also, with the limited memory, you had to be very careful about implementations as they could be making assumptions that are reasonable for a desktop, but are out of the question for an embedded device. It's pretty amazing how much the world changes at that level. However, it's getting better every year and many of the newer mobile phones are extremely powerful, thus allowing some of the constraints to be loosened.

I understand how that's an issue. For the STL, that could be overcome by defining an allocator that does sensible things for the constrained environment. But that's something the people who provide the platform should do, not application developers.

I'm only familiar with BREW at an ARM's length distance.

You were just dying to make that pun, weren't you?

[Ishamael]

I still don't follow the reasoning, but I think I know why. The STL isn't a library that you link to and gets called at run time. When you use the STL, it gets compiled in with your binary. It has to work this way because of how templates work in C++. (The definition is required at compile time; you can't just provide the interface.) So there's really no space overhead in using it, because one way or another, that data structure is going to take up space. Programs that don't use it won't have to pay for it, either.

Not exactly. It's hard to explain because things that happen in the embedded world either don't occur in the PC world or are not an issue. So I'll give an example as an explanation.

C++ name mangling caused huge issues for us with Motorola phones. Why? Because of the way the the compiled code got arranged by ARM's toolchain. There would be huge gaps, duplicated code,garbage regions, etc in the binary. This kind of stuff is invisible in the PC world, but when you've got 600K of space to work with or you're looking to use the least amount of storage possible while still writing a non-trivial application, it starts to loom *very* large. The more complexity you added, the worse it got. Needless to say, linking in stuff like the STL only added to the problem.

So there were a number of solutions, including just avoiding the STL altogether and the seemingly bloated images it caused to be generated. Another is tweaking the linker definition files. So just getting something to cross compile and run for ARM didn't mean the journey was over. In many cases it was only beginning.

Also, with the limited memory, you had to be very careful about implementations as they could be making assumptions that are reasonable for a desktop, but are out of the question for an embedded device. It's pretty amazing how much the world changes at that level. However, it's getting better every year and many of the newer mobile phones are extremely powerful, thus allowing some of the constraints to be loosened.

I understand how that's an issue. For the STL, that could be overcome by defining an allocator that does sensible things for the constrained environment. But that's something the people who provide the platform should do, not application developers.

You'd think that'd be the case and in the desktop world it's a reasonable assumption, but it's only now becoming the case in the embedded world with platforms like Window Mobile, BREW, and others. Many times, the embedded device would just provide you with some hardware, the minimal amount of "Operating System" imaginable, and *maybe* a couple of libraries to use, which would often times be buggy as hell. Our software had it's own implementation of things like malloc for example...but then again, our stuff ran below the OS, so we may have been a special case in many ways. Still, other people had to think about this stuff too.

But again, it's not nearly as bad now. MS has some really nice software for embedded development. You can practically develop the basics of an embedded OS using drag and drop tools with some of their stuff.

I'm only familiar with BREW at an ARM's length distance.

You were just dying to make that pun, weren't you?

[Nev]

I'm far too drunk to think about contributing in a heavy way to this discussion right now, but it needs to be said that standardization is near-impossible in the wireless industry as it stands, and there are a LOT of bugs in BREW on a LOT of phones (even popular mainstream models) right now.

[Kupek]

Needless to say, linking in stuff like the STL only added to the problem.

Well, my point was that the STL, specifically, <i>isn't</i> linked in. But yes, I'll accept that if you're working on a poorly supported platform with excessive resource constraints, the less variables the better.

[Nev]

That pretty much defines BREW in a nutshell!

Our software had it's own implementation of things like malloc for example...but then again, our stuff ran below the OS, so we may have been a special case in many ways.

I dunno about that. We're not redefining malloc/free, but we do have to override new/delete.

[Kupek]

I dunno about that. We're not redefining malloc/free, but we do have to override new/delete.

I think customized memory allocation is common if you're under tight constraints. My work is most certainly in user-land, but I had to write a multithreaded small object allocator because <TT>new</TT> was a huge bottleneck. (Well, it <i>used</i> to be when we were running on a Red Hat 9 system with a 2.4 kernel. When we upgraded to SUSE 9.2, which used a 2.6 kernel and presumabley an updated glibc and libstd++, <TT>new</TT> worked much better for multithreaded memory allocation.)

[Ishamael]

Needless to say, linking in stuff like the STL only added to the problem.

Well, my point was that the STL, specifically, <i>isn't</i> linked in. But yes, I'll accept that if you're working on a poorly supported platform with excessive resource constraints, the less variables the better.

Maybe, I should have used the more all-inclusive "build" rather than the more specific "compile" and "link". Sometimes I forget you're pursuing a graduate degree in computer science. So go back and substitute "build" for "link" in all of my posts and hopefully it becomes clearer.

But on the more technical point and since you mentioned it, are you sure the linker isn't doing anything when "building"(caught myself there) code that references template libraries? It seems curious based on what I (think I) remember about building programs using the STL and the fact that none of the STL code is present in the code you've written. Thus it has to be found somewhere and usually that's the job of the linker. Seems like the linker would have to play a part here, templates or no templates.

But I've got to admit, this is a bit of an esoteric area where the difference has never made any practical difference for me and I haven't used it in a long time, so I could be wrong. But it is kind of an interesting technical point to note.

[Ishamael]

That pretty much defines BREW in a nutshell!

Our software had it's own implementation of things like malloc for example...but then again, our stuff ran below the OS, so we may have been a special case in many ways.

I dunno about that. We're not redefining malloc/free, but we do have to override new/delete.

Interesting. So how do you like BREW overall?

[Kupek]

Maybe, I should have used the more all-inclusive "build" rather than the more specific "compile" and "link". Sometimes I forget you're pursuing a graduate degree in computer science. So go back and substitute "build" for "link" in all of my posts and hopefully it becomes clearer. ;)

It does make more sense. Using "link" means that the compiler has already produced object files, and some of those object files might make references to code in other object files. It's the linker's job to take all of the different object files your program depends on and make a sensible binary. And then, in that binary when you say "printf", it knows where to find that definition in the runtime system. Linking is a rather different beast than compiling, and, unfortunately, the technology hasn't changed much in the past twenty years.

Oh, and I think that "build" is a concept introduced by (Windows?) IDEs. I used to think of it as building when I worked in Visual Studio, but now that I work exclusively in a Unix environment, I never do. I could be wrong, but I just never see the term anymore.

So, yeah, being a CS graduate student means that I'm precise with my language - perhaps even pedantic. :)

But on the more technical point and since you mentioned it, are you sure the linker isn't doing anything when "building"(caught myself there) code that references template libraries? It seems curious based on what I (think I) remember about building programs using the STL and the fact that none of the STL code is present in the code you've written. Thus it has to be found somewhere and usually that's the job of the linker. Seems like the linker would have to play a part here, templates or no templates.

As far as I know, STL code gets included directly in your object files and resulting binary executable. Simply, the C++ standard does not allow a template to be declared in one place and defined elsewhere. This might change in C++0x, but as of now, it can't be done. This implies that the STL can't be implemented as a runtime library. So, necessarily, the STL exists entirely in header files, and when you <TT>#include <list></TT> it's inserting the code for an STL list directly in your code - just as if you had done <TT>#include "my_list_implementation.h"</TT>. Poke around <TT>/usr/include/</TT> (or whatever the equivalent is on Windows) and you can read the source code for the STL itself. Granted, the STL will make calls to runtime library routines at some point, but no more than your own implementations of the same data structures and algorithms would.

But I've got to admit, this is a bit of an esoteric area where the difference has never made any practical difference for me and I haven't used it in a long time, so I could be wrong. But it is kind of an interesting technical point to note.

I've read <a href="http://www.amazon.com/exec/obidos/tg/de ... 6">this</a> and <a href="http://www.amazon.com/exec/obidos/tg/de ... 6">this</a> several times over, and programming languages in general is a research interest of mine, so to me, it's more than just a technical point, but necessary in understanding the language.

[SineSwiper]

Bah, C++ talk makes my head hurt...

[Nev]

That pretty much defines BREW in a nutshell!

Our software had it's own implementation of things like malloc for example...but then again, our stuff ran below the OS, so we may have been a special case in many ways.

I dunno about that. We're not redefining malloc/free, but we do have to override new/delete.

Interesting. So how do you like BREW overall?

Meh. It's okay.

I disagree with some things that I don't really have the time to get into tonight. One of the big ones is their methods of accessing their interfaces. They have an interesting system that I disagree with mostly because it seems designed to stuff object-oriented functionality into C syntax - my feeling is you might as well just code the damn thing in C++ to begin with if you're going to go that route - you can look it up on Qualcomm's BREW site if interested; check out the IXXXX_YYYY function naming syntax, where XXXX is the name of the interface and YYYY the name of the function call, and you'll see what I'm talking about.

Also, a lot of the framework isn't well supported on some handsets (read: some of the function calls are buggy as shit on certain phones), which more the OEMs' issue than Qualcomm's, but it's still a pain in the ass to work with. Finally, some non-code-related but still annoying issues: 1. The searching tool on the BREW forums is absolutely ass, which is frustrating if you're trying to track down an obscure problem. 2. BREW Support (when I was at JAMDAT) was notorious for not looking very deeply into the issues I would send them and taking forever in certain cases.

There are a lot of good things about it too, though. I'd boil the good points down to "It's a standard in an industry without standards and most of it works most of the time." But I dunno how much more I want to get into it all tonight.

As an aside, my business partner keeps telling me that compared to J2ME, BREW is quite civilized.

[Nev]

Also, as far as the STL goes, the reason I'm a little squeamish about code size is that, as far as I know, it *does* get added to your binary - all of it. You can't just compile in, say, the one structure you need or plan to use.

I don't know if a modern optimizing compiler/linker might optimize out the unused function calls in the final binary, but I haven't really needed to try to find out, since the abstract collection class I've implemented seems to be working fine so far, and I know firsthand how tiny it is (about 300 lines of code all told, including comments.) It might be worth looking to see if the STL is worth using in the future, but right now it just seems like more of a pain in the ass to go in, take out all the namespace stuff, check code size when it's done, and so on.

[Kupek]

Also, as far as the STL goes, the reason I'm a little squeamish about code size is that, as far as I know, it *does* get added to your binary - all of it. You can't just compile in, say, the one structure you need or plan to use.

I'm confident that's not true - it runs contrary to my understanding of how templates work and contrary to the design principles of C++.

Simply, templates are not compiled unless they're used (or explicitly instantiated, but you won't do that on accident). They're always lexically parsed, but they only go through the code generation phase if they're used. You can see an example of this by writing a syntactically correct template class or function, but which contains some compile time error (type errors are the easiest to contrive). If you don't call the function or use the class, the compiler won't complain. If it's not compiled, it certainly isn't going in the binary.

Further, even if you use a particular templated class, but don't use one of its member functions, that member function isn't compiled either.

There's also no way for "all" of the STL to be included if you only say <TT>#include <vector></TT>. There's no way (let alone reason) for the compiler to see <TT>map</TT>s or <TT>list</TT>s. However, when you <TT>#include <vector></TT> that will likely depend on some base definitions that are implementation dependent, and some standard C routines and definitions. But a good optimizing compiler can remove layers of abstraction.

When Stroustrup deisgned C++, one of the design principles he used was that you don't pay for what you don't use. If you don't use virtual functions, you don't pay for them. If you don't use inheritence, you don't pay for it. And certainly, if you don't use a class or a function, you don't pay for it. So, yes, you can selectively use whatever you want from the STL. But, there is a possibility that what you want to use depends on more than you realize, so always measure before coming to a conclusion.

You might be interested in reading the <a href="http://www.research.att.com/~bs/perform ... >Technical Report on C++ Performance</a> that was put out by the ISO C++ committee in response to the same concerns you've voiced about C++ in embedded environments. It's an interesting read. They also explicity state what's <i>not</i> good to use in an embedded environment: <TT>std::string</TT> and all of the stream classes, for example.

I don't know if a modern optimizing compiler/linker might optimize out the unused function calls in the final binary, but I haven't really needed to try to find out, since the abstract collection class I've implemented seems to be working fine so far, and I know firsthand how tiny it is (about 300 lines of code all told, including comments.)

Simplicity of implementation does not always translate to more effecient code. I've implemented my own lists and queues that only do exactly what I need them to do, and the STL versions beat mine every time. But, I work in an environment where speed is the most important metric. Code size isn't an issue.

[Nev]

Hmmm, interesting. I ought to actually pick up a good book on C++ at some point - when one is self-taught, there are gaps in knowledge...anything to recommend?

Seems that it could be worth looking into, then. Speed is also a serious issue on some mobile devices, though it depends what you're doing - if we were to make, say, a true 3D game (of which there are already some for more advanced handsets), I can imagine that speed would get REAL important real fast.

[Kupek]

Hmmm, interesting. I ought to actually pick up a good book on C++ at some point - when one is self-taught, there are gaps in knowledge...anything to recommend?

Certainly. <a href="http://www.amazon.com/exec/obidos/tg/de ... 07846">The C++ Programming Language</a> by Bjarne Stroustrup. He originaly designed and implemented the language, and is the most visible member of the ISO C++ standards committee. If you want to learn about C++, this is the book to use.

A lot of people say things to the effect of "This book is too advanced for people unfamiliar with the details of C++ and the STL, start with an introductory book." Bull. I bought the book because it was a required textbook for my OO class my sophomore year as an undergrad, and I've gained significant insight from it ever since. Intially, some parts may not be clear at first, but everything you need to know to understand those parts is presented elsewhere in the book. Often, you'll have a question about something due to a coding issue, look it up in the book, and then the insight gained from answering that question will enable you to understand other sections.

[Nev]

I've heard of that many times. I suppose it's actually to the point where if I'm going to advance my career I ought to get it.

By the way, thanks for the advice.