Tuesday, December 4, 2012

My first day with openstack---development environment setup

I want to add new extension to Openstack Quantum. Before I can do any coding, I need to have a decent development environment. The developer guide on the Openstack.org wiki does not really help much for me. With some googling, I came cross a github page:https://github.com/bcwaldon/vagrant_devstack
Here are what I did to create my first openstack development environment:
1. git clone https://github.com/bcwaldon/vagrant_devstack 

2. cd vagrant_devstack
    based on the README, I also added submodules 

    git submodule init
    git submodule update

3. vagrant up
In the Vagrantfile, it tries to download vagrant box from http://c479942.r42.cf2.rackcdn.com/precise64.box, but the file is not there anymore.  I googled again and found this one http://dl.dropbox.com/u/1537815/precise64.box. I changed Vagrantfile to use the new box and restarted "vagrant up".

After waiting for sometime, I see following error message

"Mounting NFS shared folders failed. This is most often caused by the NFS client software not being installed on the guest machine. Please verify that the NFS client software is properly installed, and consult any resources specific to the linux distro you're using for more information on how to do this."

Most likely, you have a dirty vagrant environment, try vagrant reload to see if the problem goes away.

After a good a mount of time waiting on devstack to be installed, I hit on following error:
"+ sudo rm -rf /home/vagrant/cache /home/vagrant/devstack /home/vagrant/postinstall.sh
rm: cannot remove `/home/vagrant/cache': Device or resource busy
++ failed
++ local r=1
+++ jobs -p
++ kill
++ set +o xtrace
---- End output of su -c 'set -e; cd /home/vagrant/devstack; RECLONE=yes bash stack.sh > devstack.log' vagrant ----
Ran su -c 'set -e; cd /home/vagrant/devstack; RECLONE=yes bash stack.sh > devstack.log' vagrant returned 1
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.
To understand the problem, I did vagrant ssh and login to my VM. It seems that the original devstack directory has been removed. I did another git clone git://github.com/openstack-dev/devstack.git to get the devstack. Then I manually run stack.sh. It seems that at the end of lib/nova:cleanup_nova, the function tries to clean out the instances directory and the cache directory is among the deleting directories. But this directory is really mounted from my host machine by vagrant. The quick and dirty fix for this is to not to cleanup directory

agrant@precise64:~/devstack/lib$ git diff . 
diff --git a/lib/nova b/lib/nova index fbb5a01..ae4554d 100644 --- a/lib/nova +++ b/lib/nova @@ -102,7 +102,7 @@ function cleanup_nova() { sudo iscsiadm --mode node | grep $VOLUME_NAME_PREFIX | cut -d " " -f2 | sudo iscsiadm --mode node --op delete || true # Clean out the instances directory. - sudo rm -rf $NOVA_INSTANCES_PATH/* + #sudo rm -rf $NOVA_INSTANCES_PATH/* fi }

Since I have a changed devstack tree in my VM, I don't want cookbook script to re-clone the devstack tree. I have changed vagrant_devstack/cookbooks/devstack/recipes/default.rb to not clone devstack source code by disable following commands:

execute "git clone #{node[:devstack][:repository]}" do
 cwd node[:devstack][:dir]
 user node[:devstack][:user]
 group node[:devstack][:group]
 not_if { File.directory?("#{node[:devstack][:dir]}/devstack") }

execute "git checkout #{node[:devstack][:branch]}" do
 cwd "#{node[:devstack][:dir]}/devstack"
 user node[:devstack][:user]
 group node[:devstack][:group]

I'm not familiar with cookbook, I thought that "not_if { File.directory?("#{node[:devstack][:dir]}/devstack")" will avoid to overwrite my devstack directory on VM. But it seems that my devstack directory got wipe out everytime. Anyway, quickly disable lines above,solved my problem.

After changed script, I did vagrant reload in my host directory, the vagrant reload failed immediately while the cookbook tries to relink my host public key to the VM ~/.ssh/id_rsa. The quick fix for this is go to vagrant-openstack/recipes/cache.rb and fix following commands

execute "ln -s /home/#{u}/.host-ssh/id_rsa /home/#{u}/.ssh/id_rsa" 
    user u
    group u
not_if { File.exists?("/home/#{u}/.ssh/id_rsa") }
Again, I thought not_if statement should avoid the file relink is id_rsa is there already. But it seems that not_if does not work.

After all these changes, my vagrant up finished successfully. I have devstack up running. By default, the devstack does not enable quantum. But it is what I really need. To enable quantum + openvswitch, I did following changes:
diff --git a/stackrc b/stackrc index 01e9556..f47ad40 100644 --- a/stackrc +++ b/stackrc @@ -14,7 +14,9 @@ DATABASE_TYPE=mysql # ``disable_service`` functions in ``localrc``. # For example, to enable Swift add this to ``localrc``: # enable_service swift -ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-net,cinder,c-sch,c-api,c-vol,n-sch,n-novnc,n-xvnc,n-cauth,horizon,rabbit,$DATABASE_TYPE +ENABLED_SERVICES=g-api,g-reg,key,n-api,n-crt,n-obj,n-cpu,n-net,cinder,c-sch,c-api,c-vol,n-sch,n-novnc,n-xvnc,n-cauth,horizon,rabbit,$DATABASE_TYPE,quantum,q-svc,q-agt +Q_PLUGIN=openvswitch

After restart devstack, the quantum is up running.

Now on my host directory, I have a cache/stack directory that has all the devstack source code. The /opt/stack on the VM has a running copy of the devstack. I did a git branch work; git checkout work on all the directories under cache/stack so that we can have a dev branch.

Next, I will need to find a IDE to do my developemnt. eclipse+pydev+egit seems a good choice for me. First. I installed eclipse, then from http://pydev.org/updates, I installed pydev. From http://download.eclipse.org/egit/updates, I installed egit.
I used egit to import all the git repos from cache/stack directory. And then convert each of repo into general project. Switching to pydev perspective, right click on project names that just imported, select PyDev and set the project as PyDev project.

That is it, next, I will need to find out what "not_if" means and how fix the cookbook script so that I can make a right patch.

Saturday, December 26, 2009

Ethernet support for Eclair

I have created two patches for Ethernet support in Eclair. I don't want to check them in until I finish a complete test. For the early access, you can download it from the download section in android-x86.org

BTW--Chih-wei has also posted a nice video on youtube to demonstrate a really fast fastboot of Android-x86 on EeePC 901. You can find the video from http://www.youtube.com/watch?v=QeSg7S


Saturday, November 14, 2009

release of android-x86 20091113 test build

The Android-x86 project is glad to announce a new test build
20091113 for public testing. A live cd ISO and USB image are
available from our site:


Build 20091113 is Android 1.6 (Donut) based and it is a release candidate for android-x86 1.6 stable release.
In addition to the features available in previous release, the new build contains

* Hardware OpenGL support from olv in 0xlab.
* Fn key support for EeePC platforms.
* External USB bluetooth support.
* NDK for X86 platforms is supported (based on the original work from guillaume.etievent@laposte.net)
* new touch features to simulate home/menu/back keys to support touch only device.
* Improvement on the keyboard support. User can use ctrl+c directly from terminal simulator now.
* A lots of bug fixes.

Released Files:
* Live CD iso: android-x86-20091113.iso

sha1sum: dec0b99a6fe1e33e05b665e82f3020f612eb14bf

* Live USB image: android-x86-20091113_usb.img.gz

sha1sum: ce040ced2da92c278c7df2527b5f3498a68afbfa

Source code:
The source code is available in our git server:

$ repo init -u git://git.android-x86.org/android-x86/platform/manifest.git
$ repo sync

Test reports (success or fail) are welcome.
Please send the reports to the Android-x86 discussion group.

Thursday, October 8, 2009

compile openssl for mingw

I was googling a correct step to build openssl with mingw. I followed different posts from different people, none of them give me anything good. After a whole afternoon of trying and failing, eventually, I got it right with following steps:
1. install cygwin on my PC. To avoid too much trying and failing, I have installed the full cygwin. Storage space is cheap but time is very expensive.
2. install mingw
3. download openssl from openssl.org. I used 0.9.8k
4. untar/unzip 0.9.8k package to cygwin how directory
5. start Cygwin xterm (ya, Cygwin not msys)
6. in the xterm, go to openssl-0.9.8k directory that you just unpacked
7. ./Configure mingw
8. make

The steps above create static library that can be used with mingw. To create the dll, you need to do:
$ echo EXPORTS > libcrypto.def
$ nm libcrypto.a | grep ' T _' | sed 's,^.* T _,,' >> libcrypto.def
$ dllwrap -o libcrypto.dll --def libcrypto.def libcrypto.a -lws2_32 -lgdi32
$ echo EXPORTS > libssl.def
$ nm libssl.a | grep 'T _' | sed 's,^.* T _,,' >> libssl.def
$ dllwrap -o libssl.dll --def libssl.def libssl.a libcrypto.dll
After this, you can copy the *.a to /mingw/lib and *.dll to /ming/bin

Saturday, September 12, 2009

The wifi problem has been fixed

Recntly, people reported issues with wifi. The bug really has nothing to do with the wifi. It is a memory corruption issue inside the system property manager. Long ago, I changed the size of system property name to support user configure-able battery sysfs path. This change will cause the size of shared memory for the system properties to increase. But the original code of shared memory creation uses a hard coded size and I was not aware of that. When the amonut of the total used system property entries increased to certain level, segfault will happen since system property entry may have pointed to a wrong address. After adjusted the code to create the share memory, it seems that the problem goes away.
If you are still seeing the issue, please report it in the discussion group.

Thursday, September 3, 2009

What is next

Now, we have a realtivly stable system. Here are some items on my to do list:
1. enable vold to support external stroage by using fuzzy matching
2.In Android, by default, mkdir set the new directory permission to 0700. When install an new apk from web ui, the download provider tries to create a new directory called download in /sdcard. On G1 phone, the external storage is using vfat partition. And it does not enforce permission. So we can download and install the apk without any problem. But with android-x86, the external storage can use either ext2, ext3 or vfat, with the unix paritions, the permission will be enforced. So the apk install will fail. I may need to add new code the fix the permission.
3. hardware cursor, this item in my to do list for a very long time already.

Tuesday, August 25, 2009

The Android Etherent State Tracker Cont.

Ok, I have committed the first release of the etherent state tracker for the Android. You can check out code from git.android-x86.org by using branch name eth-nm-dev-br. It is an alpha release. Please report the issues to the android-x86 group