During the year 2016, I had the chance to work together with Prof. Bill Manaris on an IBM funded project to create a prototype for a cloud-based Python environment. This article summarises my view and some of the experience gained through the project.
Python has a long history of being used to access large sysems, and glue together various software components or applications. Moreover, Python’s simplicity suggests that it might be an ideal candidate to provide relatively easy access to more complex systems.
One instance of such large and fairly complex systems is given by cloud computing. Having an application run decentralised, with parts being executed on remote servers, and other parts on the local client, deviates from our traditional view of an application as a monolithic piece of software. In particular, writing such a cloud-based application requires to take into account the communcation within the application itself, deciding which parts to run where, how to handle and protect data, and, in many cases, how to properly balance the workload. For beginners, this is a daunting task, and the fast pace of evolution in this field does not help in getting to grips with it.
The advantages of cloud based computing, however, are not limited to accessing your data from various (mobile) devices without the need for explicit synchronisation. It is also the notion of accessing massively enhanced compute power. In the case of IBM’s cloud platform Bluemix (now simply called IBM Cloud), this means, for instance, accessing the resources of their cognitive computing (IBM Watson). In short, the services provided by Bluemix allow you to perform tasks like extracting information out of text and speech, gathering weather data, translating texts, or performing visual recognition. Is it possible to make all that available through a simplified interface, so that even beginners can start leveraging cloud services?
After having written a JVM-based Python development environment with a focus on simplicity (the Jython Environment for Music, or JEM), we were interested in combining our experience in writing a beginner-friendly interface with the new possibilities IBM’s platform Bluemix offers. This project was funded by IBM. The result was a prototype application, running on Bluemix, and exposing some of its services as simple Python libraries. While the original prototype on Bluemix is no longer available, we have ported a lightweight version to our own servers — naturally without the benefit of accessing any Bluemix services.
Since JEM, our original Python environment, is heavily based on multimedia interaction (music, in particular), we opted to use a client-side Python interpreter. Skulpt fit our bill, as it runs entirely inside the browser, uses Python 2 syntax (as does Jython, upon which JEM is based), and is adaptable to work with our own JavaScript-libraries. We wrote a thin interface layer to connect MIDI.js with Skulpt, and integrated everything with the ACE editor. This gave us, more or less, the basic functionality we needed to run JEM inside the browser, and make it a client-based online application. Of course, we also ported further parts of JEM’s extensive libraries to our system.
Hosting this client-based application on Bluemix did not really give us any advantages compared to running it on any other http host. A real cloud application should also provide means to save and access your own data, for instance. Yet, we focused on another aspect, and made some of Bluemix service’s available through our Python interface.
The Python libraries that ran inside the browser were just a thin wrapper that redirect all requests to our server application. The server application would make sure the request were properly formatted, invoked the actual Bluemix service, and then format the output before sending the results back to the client. When receiving the client request, we made sure the amount of data did not exceed certain limits, since we did not want to provide full fledged access to Bluemix’s services for free. Before sending the data back, we simplified it to make it more accessible for beginners.
For instance, the results of visual recognition would typically return a tree based data structure. Out server application transformed that data set into a simple list, where each item would be a list of tuples, giving a text specifier, together with a probability. After all, our goal was to come up with an interface simple enough for beginners, which typically have no knowledge yet of machine learning or sophisiticated data structures.
Here are two examples of using our interface to access Bluemix services. The first one illustrates using the translation service:
from ibm.watson import *
translator = LanguageTranslator()
translator.setSourceLanguage("french")
translator.setDestinationLanguage("english")
print translator.translate("bonjour", "bienvenu", "au revoir")
The second example illustrates using visual recognition:
from ibm.watson import *
from gui import *
picture = "tiger_1.png"
image = Icon(picture)
display = Display("Visual Recognition", image.getWidth(), image.getHeight())
display.add(image)
visualRecognizer = VisualRecognizer()
answers = visualRecognizer.recognize(picture)
for answer in answers:
print "%s (%g)" % (answer['name'], answer['score'])
The server-side application was written using the Play-framework, which certainly is a very powerful system. This choice, however, came with a few problems on its own. On the one hand, Bluemix had no direct support for the Play framework (in contrast, for example, to Heroku), so the correct configuration took some extra time. On the other hand, much of the documentation regarding the Play framework was out of sync with the framework itself. In other words, we constantly kept running into deprecation warnings with hints to use another (yet so far almost undocumented) feature or system instead.
Conclusion
Thanks to recent advanced in the JavaScript engines of modern browsers, porting an application to the web can be as simple as recompiling it to JavaScript and have it run in the browser. However, the true power of web applications lies in leveraging powerful server services, and, of course, in making data available on a multitude of different devices.
One problem, however, comes with protecting your services against unauthorised or overexcessive use. This means that the execution of a the Python programs will have to be connected to a specific account, and cannot just be openly available on the web (unless specifically chosen by the programmer). Having had limited resources at our disposal, we were accordingly restrictive in limiting data bandwidth.
Yet, the primary problem lies in the complexity and fast evolution of the involved software and services. For instance, even though the Play framework is a very powerful suite, its API is clearly not stable yet, and hence relying on the software, eventually, means to constantly update and rewrite parts of your own code, sometimes with severly limited documentation of how to do so. Getting to run the application on Bluemix is equally challenging, as most documentation assumes prior familiarity with, and knowledge of the system. That this does not need to be the case is beautifully shown by Heroku, which guides the newcomer in an excellent fashion through the steps necessary to bring your application online. However, once this initial huge hurdle of getting into Bluemix is taken, using its services becomes relatively straight forward, and a vast field of possibilities opens up.
While we found that providing a simplified interface for IBM’s services is indeed feasible and has great potential, getting up to speed with all the technologies and systems required for writing a cloud application already exhausted our resources. Making sure that the application scales well, stays up to date, and adheres to the necessary security considerations was beyond the scope of this project.
