Wednesday, February 24, 2010

OSGi: Visualizing system complexity


For a while I have been experimenting with visualization of assembled OSGi systems based on the bundles explicit imports and exports.

In particular, Coalevo, because there are a lot of inter-dependencies between the different services and bundles.

See for yourself:

Now, logically many of the underlying pieces are used by almost all other bundles, so I have tried to remove some of those bundles (i.e. libraries, foundation, logging, system-services etc.) to obtain a clearer picture:


Nice!

I am still working on improving the import/export parsing (because it has become way more complicated then it was in the early days of OSGi) and the way the dependencies are resolved. Currently it is rather simplistic and doesn't take into account more than simple packages and no versioning.

Credits for the actual graph and visualization go to Jung, which is an amazing graph library with easy to use visualization tools.

If anybody is interested in this, don't hesitate to contact me.

Saturday, January 23, 2010

Nano, in Shell Access 2, A Technical How-To

Given that it took me quite a while to figure out a way to make this happen, I'd like to share the solution that we have created for Coalevo (and KaraNet) to be able to edit messages using a GNU nano editor from within the Java based Coalevo platform (specifically the shell access).

Essentially, there are three key points that make it possible:
First, the ProcessBuilder, that helps to create and run a process from within the JVM
Second, handling the Streams and pumping the I/O between the JVM and the process
Third, tricking Linux into running nano as if a terminal is available

There is no big deal about the first part, actually its relatively easy to create a ProcessBuilder instance with a command, add environment variables and start a process:


//Prepare command
List l = new ArrayList();
l.add(System.getProperty(NANO_SYS_PROP));
l.add("-R");
if(m_Shell.getSession().isRebindDelete()) {
l.add("-d");
}
l.add(m_File.getAbsolutePath());
ProcessBuilder pb = new ProcessBuilder(l);

//Prepare environment
Map env = pb.environment();
env.put("COLUMNS", Integer.toString(m_Shell.getShellIO().getColumns()));
env.put("LINES", Integer.toString(m_Shell.getShellIO().getRows()));

//Start Process
m_Process = pb.start();


The second part is a little bit trickier. There are several approaches to this, but given that there is already one thread per user connection running, we decided to use this thread to pump the streams and take an approach (kudos for this one to gnodet from the Apache Mina project).


//Prepare connection streams
m_Input = m_Shell.getShellIO().getTerminalIO().getBaseIO().getInputStream();
m_Output = m_Shell.getShellIO().getTerminalIO().getBaseIO().getOutputStream();

//Acquire process streams
m_ProcessInput = m_Process.getInputStream();
m_ProcessOutput = m_Process.getOutputStream();
m_ProcessError = m_Process.getErrorStream();

//start pumping
pumpStreams();

Here is how the pumping mechanism works:

private boolean pumpStream(InputStream in, OutputStream out, byte[] buffer) throws IOException {
if (in.available() > 0) {
int len = in.read(buffer);
if (len > 0) {
out.write(buffer, 0, len);
out.flush();
return true;
}
}
return false;
}//pumpStream

private void pumpStreams() {
try {
for (; ;) {
if (!isAlive()) {
return;
}
if (pumpStream(m_Input, m_ProcessOutput, m_Buffer)) {
continue;
}
if (pumpStream(m_ProcessInput, m_Output, m_Buffer)) {
continue;
}
if (pumpStream(m_ProcessError, m_Output, m_Buffer)) {
continue;
}
// Sleep a bit. This is not very good, as it consumes CPU, but the
// input streams are not selectable for nio, and any other blocking
// method would consume at least two threads
Thread.sleep(1);
}
} catch (Exception e) {
m_Process.destroy();
}
}//pumpStreams


The third part is actually not necessary on Darwin/OS X, where the process from the JVM seems to be already providing a terminal. However, it is necessary on Linux for standard JVMs. Kudos for this trick goes to Mitch; its essentially running nano through a local relay called socat:

#/bin/bash
export LANG=en_US.utf8
socat - EXEC:"/home/coalevo/nano $*",pty,stderr


Last but not least, the nano we are using has been slightly hacked, and is always started in a restricted mode (-R flag).

For the complete picture, we suggest to check out the EmbeddedNano source code.

Wednesday, December 2, 2009

Want to help out? Read on....

We are looking for help with the development of the Coalevo
Platform!

1) Why should/would I want to help with Coalevo?
aka What is in it for me (ROI)?

Possibility to learn new things and increase your experience.
It's not a promise, but commonly more experience
will reflect in your "value" and income in the long run.

Here some related Cali Dude writeup with some related thoughts
"Top 10 Reasons To Work On Open Source":
http://karanet.at/r/7q0

2) Are there ideas of what could be done?

Proposals & Ideas can be found here:
http://www.coalevo.net/confluence/x/CQ

Here some specific ideas that you could get started with
or help us to get started with.

Statistics & Monitoring:
http://www.coalevo.net/confluence/x/DQBU

Atom Publishing Protocol Server:
http://www.coalevo.net/confluence/x/AgAm

Atom Publishing Workspace for Discussion Service:
http://www.coalevo.net/confluence/x/FQBU

IRC Group Messaging Server:
http://www.coalevo.net/confluence/x/BYAi

KeyPhrase Extraction:
http://www.coalevo.net/confluence/x/AoA5

3) Can I work on my own ideas?

SURE!!!

You may also provide ideas and see if somebody catches
fire on it and helps you out.
Just write something up adding it to Proposals & Ideas
that can be found here:
http://www.coalevo.net/confluence/x/CQ

4) Will you support collaborative development?

SURE!!!

a) Social side:
We will try to mentor newcomers and will provide them
with exposure and help to move along.

b) Tool side:

We have a top-notch set of online tools for collaborative
development: issues, wiki, VCS and code browsing, everthing
is available and integrated.
If required, we can upgrade and extend the possibilities.