The re-entrant coder

Found a great read on why Android or iPhone is the wrong question.

While Apple may have opened the proverbial Walled Garden, it is Google, with its aggressive Android offering, that aims to obliterate it. Make no mistake about it; Apple was the pioneer with the amazing revolutionary product. Also, with no iPhone, there is no Android. This is not to say that Android copied iPhone, but rather the impetus to adopt and trust Google’s Android offering was driven by a market dynamic that resulted directly from the iPhone’s success.

via Android or iPhone? Wrong Question « abovethecrowd.com.

Indeed, the iPhone represents only a very small portion of the global phone market. But it is the other portion of that same phone market that is the target for Google/Android. Google does not necessarily want to compete with the iPhone head-to-head (although they can!), but is more looking towards converting people that use a heavy feature-phone these days but may make the switch to a smartphone if a compelling enough offer comes along.

So…. Here we go again: that all too familiar feeling of ‘uhoh’…. I was working on my PC the other night, doing a routine upgrade of KDE to the newly release version 4.4, when all of a sudden aptitude started spewing loads and loads of errors about not being able to save files and giving read-only errors all over the place. A quick check with mount and dmesg confirms my suspicion: a hard drive is giving problems and the file system has been remounted read-only. Dmesg confirmed that it was the drive timing out on commands, giving mentions that it was unable to remap a sector.

Now, this puts me into a bit of a bind. This is an olderly PC that I scrapped together from a Dell Dimension 8400 and an additional HD and SATA card. As it doubles as the media-server for my XBox, it has to have some additional storage, hence the extra harddisk. The extra harddisk was a 300Gb one, the system harddisk it came with (which started failing) was a 160Gb one. I’ve routinely run LVM2 on my system exactly for this case, being able to add additional diskspace easily without fiddling with mounting additional drives, running out of space on one partition while swimming in space on another one. This time, that may have gotten the better of me though. I don’t run LVM in a RAID setup, so if one drive fails, I’m up sh*t creek. My only luck may have been that I only very recently added that disk, and I hadn’t really started using it.

So, now I need to recover this data somehow. My plan of attack is as follows:

• Put in another drive
• Move the entire contents of the failing drive to the new drive
• Try to reconstruct the LVM data from there.
• Profit!

Step 1: find a new drive

We need a donor drive to transfer the data too. Preferably, use a drive that is bigger than the original drive. Nothing would be worse when you try to rescue your precious data and finding out that you are a few sectors short on the new drive. Even if both drives are advertised as being 160Gb drive, that doesn’t mean they will have the exact same sector count! When we have the drive, hook it up to the system

In my case, I bought a 250Gb Samsung drive, only to find out after I built it into my machine that it was a) refurbished (it had a complete Vista install on it) and b) dying! I immediately got a warning from Ubuntu that the SMART data indicated that the drive was failing… Not good. I removed the drive and replaced it with a slightly larger 160Gb drive I had lying around. I will replace that drive when I get a replacement for the refurbed drive, but by then, I can hopefully use the ‘pvmove’ commands.

When connecting the drives, make sure that you hook up the donor drive to the cable that was originally connected to the failing drive and connect the failing drive with an extra cable. This way, the device enumeration under Linux would be identical and the partitions we salvage will appear on the device files that LVM expects.

Step 2: re-create the partitions you had on the drive

This step, in retrospect, may not have been necessary. I booted the mediaserver into a bootable Ubuntu 9.10 CD-image and re-created the partitions using Gparted, ensuring that each partition was slightly larger than the corresponding partition on the failing disk. Note: gparted will complain that it is not capable of working with LVM2, but you can ignore that.

Step 3: get a copy of dd_rescue

For the copying of the data to succeed, we need a copy of dd_rescue. Dd_rescue is a clone of dd, that copies data from a file or device and does not abort when it encounters a read error. Instead, it will fall back to a ’sector-at-the-time’ reading mode, log the error and continue. Additionally, it has the option of reverse-copying, starting at the end of a device and working its way backwards, so that you can approach a problem area from both sides, maximizing the amount of a file or device that can be read.

Ubuntu 9.10 does not come with this by default, but the sources are tiny and luckily our live-CD includes a working gcc compiler. Compiling it is a breeze, just consisting of a ‘make’ call. We run it is follows:

$ sudo ./dd_rescue /dev/sdc1 /dev/sdb1

Of course, you will need to change /dev/sdc1 and /dev/sdb1 into their appropriate device files. Also, be very careful about the order  of these arguments. The first is the input file, the second argument is the output file. Mess these up and you are overwriting the data you want to recover with whatever was on the donor drive!

I had two partitions, so I executed this command twice. In retrospect I could have just specified the device files for the entire harddisk (i.e. /dev/sdc and /dev/sdb) and dd_rescue would have duplicated everything (including the partition table).

I got around 40 read-errors on the first partition, and about 520 on the second partition. Now we have the drives duplicated, but don’t throw the faulty drive away just yet! If we manage to cock up the LVM restore, we can re-attempt the saving.

Step 4: start salvaging using LVM

Now we have a copy of our physical volumes. We will try to activate the volumegroups they belong to by executing vgscan:

$ sudo vgscan -v
Wiping cache of LVM-capable devices
Wiping internal VG cache
Reading all physical volumes.  This may take a while…
Finding all volume groups
Finding volume group “htpc-server”
Found volume group “htpc-server” using metadata type lvm2

After this is done and our volumegroups have been found, we execute a vgchange:

$ sudo vgchange -a y
2 logical volume(s) in volume group “htpc-server” now active

vgdisplay will give us the names of all volumes and volumegroups:

$ sudo vgdisplay
— Volume group —
VG Name               htpc-server
System ID
Format                lvm2
Metadata Areas        3
Metadata Sequence No  5
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                2
Open LV               0
Max PV                0
Cur PV                3
Act PV                3
VG Size               428.24 GB
PE Size               4.00 MB
Total PE              109629
Alloc PE / Size       109629 / 428.24 GB
Free  PE / Size       0 / 0
VG UUID               2arhGV-xkVa-MUXt-38PD-XOmE-TaYN-tpPJBT

Finally, we can execute a full fsck on the volumegroups involved and if no unfixable errors are found, we can mount the filesystems and start assessing the real data-loss.

To access the Android code from my laptop at work I need to go through a proxy. I had a hard time finding info on how to use git with a proxy, but then I found this link detailing a script that uses socat to get through a proxy. The sample script they provided did half of the work of connecting to the proxy. However, it didn’t handle the authentication part. Fortunately, socat has a few more tricks up its sleeve and adding a proxyauth option handles this case as well.  Full script can be found below:

#!/bin/sh
# Use socat to proxy git through an HTTP CONNECT firewall.
# Useful if you are trying to clone git:// from inside a company.
# Requires that the proxy allows CONNECT to port 9418.
#
# Save this file as gitproxy somewhere in your path (e.g., ~/bin) and then run
# chmod +x gitproxy
# git config --global core.gitproxy gitproxy

# Configuration. Common proxy ports are 3128, 8123, 8000.
PROXY=yourproxy
PROXYPORT=8080
PROXY_AUTH=username:password

exec socat STDIO PROXY:$PROXY:$1:$2,proxyport=$PROXYPORT,proxyauth=$PROXY_AUTH

Be aware that your proxy must allow outbound connections to port 9418! If not, this script will not work

Well, got a nice perk this week at work: assisting in getting an embedded linux board up and running. To be honest: it is a nice little piece of work, called a FriendlyArm. It is an ARM9-based development board, costing less than $150 with an 3.5″ TFT with touchscreen (optional 7″ touchscreen). It is fitted with 64Mb RAM and 64Mb Flash, and 2Mb NOR.

I’ll be trying to get this to work, so expect updates on the blog. Eventually, I’d like to get Android running on this board as well (but memory may be a problem).

Full specs after the break

Read more…

Marketshare Android US

Well, it does seem that Android is doing quite nicely on its own… Its market-share seems to have risen from nought to 5% in less than 5 months time. And that is with only 1 device out, on only 1 carrier (as opposed to Apple, who have 3 devices in the hands of 70 operators). The future is looking bright…

(via droideo.com)

We have seen Android running on the T-Mobile G1 and very soon we will see Android on the HTC Magic (by Vodafone). But… did you know that you do not necessarily need a new mobile to run the Android OS? Due to its open source nature, the Android stack is available for everyone to tinker with, and tinkering they did. At the moment ports exist for the following phones (note: these are the ports that I’m aware of)

• HTC Vogue
• HTC Touch HD
• Toshiba G900 (sorry, could find the link again)

But also for our beloved mobile, the HTC Touch Diamond a port exists. It is not yet fully functional, as a few important areas are still missing, including power-management, sound and WIFI/Cellular data. But call management, SMS and the rest of the functionality of the Diamond works just as it should.

Detailed instructions for installation can be found on Connect-UTB, an enthusiast-run site, but it is as simple as downloading a zip-file and extracting it to a directory /tmp on the Internal Storage. Before running it, you should disable your PIN-code of your SIM, as the current port is not able to handle entry of the PIN-code after starting. Then it is just a matter of running haret.exe and having some patience. The first boot can take up to 2 hours.

Progress on this port can be followed here, or at the following pages:

• http://wiki.xda-developers.com/index.php?pagename=RaphaelLinux
• http://forum.xda-developers.com/showthread.php?t=402002
• www.connect-utb.com

let me know if it works for you!

Interesting… The Wine (Wine is not an Emulator) project has been allowing me to run Windows applications on my Linux machine at home for quite some time. Recently, I needed to burn an image to a dual-layer DVD and was having problems getting it to read in my in-dash navigation system. Several posts on the internet pointed out that I was best off using Verbatim media, but I also feared that the ancient firmware on my burner (an NEC ND-3500AG) might be to blame.
Newer firmware was available, but as is almost always the case only available on Windows. As I do not have a running Windows image on this machine (let alone the possibility to run DOS) I figured I might as well attempt the flash using Wine. Should it fail, I could always take the drive out and attempt a re-flash on a Windows-based machine. What spells my surprise to find that the flash worked flawlessly! I’m not advocating this route for each and every flash, but I’ve seen more success-stories of firmware flashes using Wine.
And yes, the subsequent burn worked flawlessly…

As listed in my previous post, I’m looking at finding out more about coredumps. The page I found the info below on lists that the limit is the maximum size of a process image that a coredump would be generated from.

However, the file will not be created unless your “core dump size limit” is set high enough to accomodate the core dump file.

Unfortunately, it doesn’t refer to the Unix/Linux version used and the age of the information is also unknown. So further investigation ensues.

OK, so here’s an interesting one: I’m currently doing the architecture for a new embedded controller running Linux as its OS. It will use a SoftPLC for the actual control of the hardware and I want it to have a proper way of handling OS failures. We’ve looked at the glibc backtrace() functions and they would solve part of the problem.
Our main problem is the fact that the root fs of the controller will be on a read-only cramfs and that the writable flash partition is only limited in size. We will have more memory than flash, meaning that we cannot accomodate full coredumps. Of course, for this the ulimit -c command has been invented, but I wonder what happens if I set the limit to 1Mb for example. If the process image is larger than this, would the coredump not be generated at all? Or would the generated coredump be limited to this size? And if it is limited to a certain size, is there a coredump size you should not dip below risking that the coredump becomes useless due to missing stack information? The online documentation I found was inconclusive, so its on to experimentation. I’ll update this post with my findings…

It seems that the intrepid hackers at xda-developers are at it again. They’ve been working away on porting Google Android to run on many different types of HTC kit. At the moment, ports are underway for the HTC Vogue (a Touch for CDMA-based networks), HTC Polaris and HTC Tytn II. As I’m using an HTC Touch ( the one with the measly 64Mb of RAM) I left in the cold for the moment…

Which prompted me to see if I can run Android on my phone as well. I’ll try and chronicle my efforts here.