Merry Christmas and Happy Holidays!

To all the people that celebrate Christmas, to all my friends and their families, we wish a very Merry Christmas!

Happy Holidays!

Python 3.0 and print function

I have been following the python mailing list for quite some time now. I recommend it for any pythonista who's not afraid of a bit of spam and flame war here and there.

There has been a lot of discussion and arguments about a change in Python 3.0 in which print is a built-in function now instead of a language-specific statement. The arguments started flowing in to the mailing list just minutes after Python 3.0 final was officially released.

Most complaints seem to revolve around the fact that *now* we have to write extra two parenthesis, ie. print("something") instead of print "something". Let us ignore that complaint for now, as it is minor (not to mention pointless) issue, something to get used to and easily solved with IDEs. Besides, we all *know* how to use functions, right? This is a non-issue.

Another (much larger) complaint is about string formatting (PEP-3101 - Advanced String Formatting), which is a very big improvement over the formatting using "%". New built-in function, format(), has been introduced along with (so-called) "Format Specification mini-language" to specify and control formatting. Quite powerful, if you ask me.

Particularly interesting formatting option is formatting of positive and negative numbers (a sign option). Consider implementing this in the old syntax:

print(format(-123456,'#> 20'))
print(format(123456, '#> 20'))
print(format(123456, '#>+20'))

Furthermore, you can implement __format__() method in your classes, which will be called by the format() built-in function for producing a formatted version of the object. However, implementing formatting specification is up to you - you can use the standard formatting syntax or implement your own.

Bottom line is, Python 3.0 brought a lot of changes and improvements; those changes have been discussed and polished over a period of several years prior to releasing the final version. What I don't understand is those people who are complaining about it now, instead of doing it before and contributing to those discussions. It will get some time to get used to the new changes, sure; but don't dismiss them straight away - give it a chance and try it out. You will soon realize that those changes are in fact good ones.

Python 3.0 released

Interestingly, my previous post was about Python 2.6... This one is about 3.0, which finally saw the light of day in final edition two days ago! I just noticed it now (obvious proof I'm spending too much time on meetings here at work :).

Sadly, no official package for OS X as of yet, but I'm sure it will arrive after the weekend. I'm itching to take it for a spin on my code.. There will be some incompatibility issues, but 2.6 version helps a lot to sort that out.

Have a nice weekend, folks!

Python 2.6 final

It's here, the final version of Python 2.6 has been released. This is the last 2.x release of Python before the almighty Py3K. You can read what is new in Python 2.6 and of course, grab your own copy.

I'm fetching the installation now, eager to see if the installation will work with PyObjC on my Mac. Next thing on the list is to install Stackless.

Apple dropping NDA for iPhone Developers

I visited Apple Developer Connection site earlier, looking for some iPhone related resources. There was a message posted there, for all iPhone developers: Apple has decided to drop the non-disclosure agreement for released iPhone software!

In a released message, Apple stated that they have introduced the NDA to protect many of its inventions and innovations that have been put into the iPhone OS. They (finally) recognize that this has put a huge burden (and lots of problems) on the developers so they have decided to drop the whole thing.

However, this agreement is dropped only for developers who have released their software on the Apple Store, the NDA still covers the unreleased software until its release.

This will make a lot of developers much happier. Very good move by Apple!

World is concurrent

Last few years or so, in my free time, I have been coding in Stackless Python. I'm using Python almost exclusively for my projects, so it comes natural to use Stackless for concurrency. Concurrent programming is very interesting and challenging, and Stackless Python makes it (very) bearable and easy.

Stackless Python introduces the concept of microthreads, where tasklets wrap functions allowing them to be launched as microthreads. Scheduling is built in and it can be either cooperative or preemptive. Finally, there are channels which can be used for communication between tasklets. Channels block tasklets, either receiving ones or ones sending, depending on if there is a waiting receiver (or sender, respectively). Another interesting thing is that tasklets can be serialized (pickled) to a disk (or any other storage media) and deserialized later on to be resumed.

Using this functionality provided by Stackless (through a single module) is very easy and intuitive. It keeps the Python code very readable and understandable and it even improves the structure of the program. Common usage patterns are available from the authors of Stackless to help people new to concurrent programming understand principles of how Stackless is used. It allows creating custom tasklet functionality (ie. named tasklets), as well as custom channel functionality (ie. broadcast channels, sending of messages with a timeout, etc).

As I mentioned, scheduling is built in and it is up to a programmer to choose the type: preemptive or cooperative. With cooperative scheduling one has to be careful to write code so that tasklets run cooperatively. During the design and implementation of the tasklets, programmer should pay attention to run the scheduler manually if the operation within the tasklet might make other tasklets suffer for not being able to run. On the other hand, with preemptive scheduling, the scheduler itself is configured to interrupt and reschedule running tasklets. I found the cooperative scheduling more useful in my implementations since it gives more control. You can, however, using preemptive scheduling kindly ask the scheduler not to put your running code back to the scheduling queue. One useful idiom is the Sleeping Tasklets, which blocks the tasklets for a certain period of time. Interestingly, that idiom uses channels to accomplish this.

Documentation of Stackless is available on the website and it covers basic functionality. Also, it offers examples and common patterns (idioms). The module itself is very well documented and available in the python interactive shell by typing help(stackless). The community is active on the mailing list, where help is always available. Every now and then there's a good discussion about the advanced usages of Stackless, and I highly recommend subscribing to the list.

Issues exist, however. Current CPython implementation suffers from the infamous GIL (or, Global Interpreter Lock) which makes it difficult for Python to fully utilize multicore systems (almost every recent computer nowadays). For those who don't know the effects of GIL, it is a mechanism to keep multiple threads from modifying the same object at the same time. Only one thread that acquires a lock on the object may modify that object, and the interpreter controls the acquiring and releasing of the lock, especially around potentially slow operations (such as I/O).

There has been quite a lot of talk about the GIL and whether it should (or not) be removed from Python. Back in '99, few brave souls (Greg Stein and Mark Hammond) tried and removed GIL from the Python code using locks on all mutable structures. According to benchmarks, it actually slowed the execution of code. The results showed that the execution was twice as slow in single threaded environment than with the GIL. If ran on multi-CPU (multi-core) systems, there would be no actual performance gains by removing the GIL.

To truly make the code distributed between CPUs, the solution is to run several python processes and communicate tasklets between them, however that can get (very) complicated, to say the least. Luckily, Python 2.6 (next major release of Python, with final release just around the corner) comes with multiprocessing module. This module supports spawning processes in a similar fashion threading module is used. More so, since the module follows the same API as threading module, it makes refactoring of your projects which use threading a breeze. Process objects can communicate between each other using Queues or Pipes, the latter being bidirectional (both parties represent ends of the pipe, with send() and recv() methods available for sending and receiving).

Using multiprocessing module, you can even distribute your processes remotely, and on top of that, it is possible to create a pool of workers for running tasks. Finally, since the module uses subprocesses instead of threads, the effects of aforementioned GIL can be circumvented.

Stackless by design (because of the scheduler) does not allow tasklets to access (and modify) data structures at the same time. Moreover, in true concurrency fashion, it utilizes channels for passing the data between tasklets. It still has a notion of a shared state between tasklets, but of course, without the dangers. With multiprocessing on Python 2.6, Stackless Python programs will be much more scalable than they are currently, utilizing multi-cpu and multi-core environments more efficiently. That still, however remains to be seen, as porting of Stackless to Python 2.6 is a work in progress.

Enter Erlang! For those of you who don't know what it is, Erlang is a functional programming language (an entirely different approach to programming compared to, ie OOP) designed in Ericsson for the purpose of developing concurrent, distributed, fault-tolerant and (soft) real-time applications.

Being functional, it does not have a concept of a state, it has single variable assignments (just as they taught you in Math classes), dynamic typing, pattern matching of functions (with guards), etc. On top of that, it has extremely lightweight processes with no shared memory. Processes pass messages around to communicate between themselves. Erlang runtime supports very high number of concurrent processes and they are actually abstracted from the operating system. It also supports dynamic distribution of processes, both locally (over multiple CPUs or cores) or remotely (over the network to another Erlang runtime node). Thus, you can build large scale distributed applications that run on machines with many CPUs and on many machines in the network. Since there's no shared state or shared variables, all traditional problems related to concurrency simply disappear as the need for locks is removed.

Erlang does give a lot of headache to a lot of people, because of its syntax. Comparing the syntax with Python's, I can say (being biased and all) that I definitely prefer Python's. On the other hand, I quite like the syntax of Erlang, too. It looked quite bizarre at first, but going through the documentation and examples helped me understand the basics of it. I have just got delivered the Erlang book I purchased, and am very excited to learn more about Erlang. As I have always been more interested in building backend applications, using Erlang seems like a good choice worth learning more about.

As a follow up, next article on this topic will be sprinkled with some code examples, both in Stackless Python and Erlang. I really like learning about new programming languages and frameworks by actually implementing something useful using them, so I will try to do so with Erlang. Some ideas are popping into mind and in next articles I might elaborate more on that, too.

Django on Jython

Few days ago, Leo Soto announced on jython mailing list that Django now runs unmodified on latest Jython code. This is great news for both Jython and Django projects.

Although not my first framework of choice for developing web applications using Python, Django has been adopted by many developers out there, even few ones defecting from Ruby on Rails. Django follows the MVC design pattern, it has url patterns similar to Routes in RoR, etc. One slight problem for developers might be the fact that Django has it's own database API - although I believe another ORM such as SqlAlchemy can be used instead.

I have been following the development of Jython very closely, mainly because I'm planning to introduce it in my company. Our software policy states that the main development language is Java and I'm trying to push the initiative that enables us to use any language as long as it runs in JVM. Nowadays there are several very good (and strong) candidates to replace Java and Jython stands IMHO as one of the strongest. It takes the strengths of Python language and brings them into JVM.

We adopted the Spring Framework in our development to help us minimize the Time to Deliver and maximize the ROI. For developing web applications we're using Spring MVC in both portlet and servlet environments. It improved both the quality of our deliveries and shortened the time to deliver. However, there is still the huge overhead of the code compilation in our day-to-day development. Something like Jython (and Django, possibly) can eliminate that problem partly (or entirely).

Django developers are going for a 1.0 release early next week. With that in mind, the fact that there's an excellent Django Book online and that it runs in Jython are more than enough reasons to get to know Django better and (even) start using it in Enterprise environment. Why not?

Python, the evolution

I just stumbled upon a project called code_swarm, which is an experiment in organic software visualisation. The project takes information from a source control system and creates a visual representation of the history of code commits. The end result looks quite awesome. Below is the code_swarm video of Python source code and its evolution.



The project's source code is available here, free of charge, of course. It is created by Michael Ogawa, using Processing environment. Great stuff!

Protocol Buffers

Google decided to open their tool for serializing structured data, called Protocol Buffers. It's language and platform neutral way of communicating data over networks or serializing it for storage. Interesting bit is that Google is using it in almost all of their projects.

On the surface, tt reminds me a lot of CORBA, especially the way you define message structures, but it differs from it a lot in terms of message exchange and serialization - you can store and/or communicate your data structure across the network by any means, unlike CORBA where you're forced to use CORBa message brokers. In my opinion, that's the main reason why CORBA was never so widely accepted.

How does it all work? First of all, you define your structure using a DSL and store it in a .proto file. You then compile that file using a tool and create data access classes for your language of choice. Classes can be generated for C++, Java and Python - my choice would be, as always, Python. Those generated classes are then used to create, populate, serialize and retrieve your protocol buffers messages.

The messages are very flexible - you can add new fields to your messages without breaking old code that's using them; they will simply ignore it. That functionality comes in very handy, especially for larger systems (think versioning and deployment).

Yes, but what about XML, you might ask? According to Google, PBs have many advantages over XML; they are:

• simpler
  
• 3 to 10 times smaller
  
• 20 to 100 times faster
  
• less ambiguous
  
• generate data access classes that are easier to use programatically
  

I might disagree with the last one (think of JAXB in Java world), but I completely agree with the other ones - it's quite true that XML tends to be cumbersome and a big overkill, especially in environments where size and speed does matter.

For the end, I saved a very interesting quote from Google's PB pages:

Protocol buffers are now Google's lingua franca for data - at time of writing, there are 48,162 different message types defined in the Google code tree across 12,183 .proto files. They're used both in RPC systems and for persistent storage of data in a variety of storage systems.

Looks very interesting and very promising, considering Google is behind it. I'll follow up this article with some neat examples.

Back at work after a long vacation

I was away on vacation for 7 weeks. Now I'm back at work. Tough, I must say. My mailbox is a total mess, loads of new mail, meeting invitations, updates and changes, etc. Then an idea hit me! Our corporate email suite comes with Out of Office assistant, which helps you (and your coworkers) to manage the email while you're away on vacation (sending out auto-responders and such). It also has a very nice feature of reporting who emailed you and all that. I am still going through the emails I received, totally stressed out not to miss something important. I'm skimming through as fast as I can and it seems there's no end to it. I'm quite sure I missed something very important, although I'm trying my best to mark messages that I need to check again later. That means, of course, that I'll be reading my emails twice.

Now, the idea - wouldn't it be great to have a feature which would, upon opening your email suite after a long vacation, populate your inbox little-by-little. Let's say you have 120 messages which have been sent to you while you were on vacation. The suite would sent you a dozen messages every 10 minutes, so you can slowly read it and catch up. Preferably over a good cup of coffee. No stress, no worries, you can read your mail at a normal rate as you would if you didn't go on vacation. I'm sure it would take some people few days to catch up, but it sure beats reading 500+ emails in one hour.

Back to reading emails.

Core Spring Course

I'm attending a Core Spring Training Course this week, held and organized here in Iceland by my company. I have finally managed to persuade my superiors to invest into Spring training for all the employees of our department. The course is held by Arjen Poutsma, the developer behind Spring Web Services project.

While I have been developing applications with Spring Framework for past few years, I am finding the Core Spring training course very informative and helpful. Most of the material covered so far is familiar and well-known to me, apart from the more advanced usage of Spring AOP, which I haven't used much so far. That is definitely going to change, I am sure. Every topic is followed by a very well designed lab where the newly learned material can be exercised.

Things to cover in the next two days are Spring MVC and Web Flow, configuring Spring Security, Remoting with Spring Web Services, etc. I sure hope we'll be covering (soon to be released) Web Flow 2.0 - I promised to write more about it when we start adopting it in projects, but we have not reached that point yet (one of the reasons is the fact that Web Flow haven't reached final 2.0 version).

While I prefer figuring how things work by experimenting on my own, it will be great to get information about Web Flow first-hand through the presentations and lab work on the course. Which brings me to the point -the real advantage and real value of this course is the chance to talk in person to people behind the Spring Framework, hear their opinion on things and get some good tips and advices (especially advices on AOP advices). Labs are in fact designed with that in mind.

I will write more about Spring AOP and Web Flow in the following days, so stick around.

Spring MVC easy way

Writing web applications using Spring MVC has never been easier, now that we have annotated controllers in Spring 2.5. Basically, it all boils down to just couple of things, from defining the viewResolver bean and the required XML configuration to scan for annotated based controllers to adding an annotation to (any) method in your controller class, just like this:

@RequestMapping(value="/url")

public ModelMap method()

Of course, there are few required beans you need to define for all that magic to work (shown in the Spring Framework documentation online) but I find that very acceptable and a big plus (showing how Spring is very flexible).

Some people might argue that the (now) old-fashioned way of defining the links the controllers respond to via urlMapping bean (SimpleUrlHandlerMapping) is similar to routes in Rails where everything (in regards to URL requests) is in one place. Then again, how many times are you going to change the URL locations and where does it look to be more natural - especially when you basically do all your work within the controller.

Similarly, this is how TurboGears (a web framework for my favorite language, python) handles request mappings (with a twist), where the name of the method in the controller declares the request mapping. That can be easily achieved with Spring MVC annotated controllers, too.

All this ease-of-use for writing web applications plus all the benefits of the Spring Framework - there's no need to switch to another framework, developers used to work with Spring can leverage all their knowledge and still be in the front lines, especially in regards to developing web applications.

Jython development gaining momentum

Jython developers gathered last Sunday in San Francisco and held a sprint to work on the next major release. This will bring Jython implementation of Python on par with the CPython implementation, which is now version 2.5.1 (with a v2.5.2 release candidate 1 released to public testing just few days ago). The San Francisco Sprint focused on the Roadmap list for the Jython 2.5.

There is not much information to work with in regards to what is the outcome to the Sprint, but I suppose a lot of good work came out of it. Hopefully we will see a stable release soon. With Groovy and JRuby already having a steady (and growing) number of followers, Jython needs a jolt.

Technical evangelist?!

What is up with technical people and the urge to have titles? Especially strange ones.. I have been reading a lot lately about Flex and AIR, technologies for developing rich internet applications, mainly for two reasons - first and foremost, it's very interesting technology and second, it might come in handy at work. But I'm not going to talk about that just yet.

During my reading, I came across more than a few times with a a rather unusual title. Apparently, one of the guys behind Flex and AIR calls himself a technical evangelist. If we look up the definition of the term evangelist, we can find out various things:

• Evangelism is the verbal proclaiming of the Christian Gospel or, by extension, any other form of preaching or proselytizing.
  
• "To announce the good news", one who preaches the facts of the Gospel in order to win converts.
• The traditional view of the evangelist is a bearer of the "Good News", proclaiming the gospel to the unbelieving world.

So, technical evangelist is doing what, exactly? Somebody who verbally proclaims technology? Isn't just pretty much everyone who works in the field an evangelist? We should stop calling ourselves bloggers and start using evangelist instead..

This is, in my honest opinion, the worst title I have ever heard in my life. Sounds ridiculous and I honestly don't believe how can anyone take it seriously. Rubbish.

Street view..

Funny, yet a bit scary video portraying usage of Google Maps and Street View. Check it out below..

Few years from now, who knows..

Shift some time

Few weeks back, I downloaded a TimeShift game demo from Xbox LIVE! Marketplace. The demo features both single-player (campaign) and multi-player modes. The game itself is a first-person shooter game but unlike many other similar games, it has a rather unique feature - playable character in the game wears a special suit which enables you (as a player) to manipulate flow of time in the game.

You are able to slow down, stop and even reverse time, which affects everything (and everyone) around you - while you move freely! Some people might remember the very popular Max Payne series of games with a special (and very cool matrix-esque) Bullet-Time feature - you could put things in slow-motion to aim better and deliver bullets precisely.

In TimeShift, you're able to do that, and a lot more! If you get flanked by enemies, no problem - reverse time a bit and throw a grenade where they pop up - problem solved. If there are too many enemies to handle - stop (or slow) time and give each of them a proper gift (a bullet in the head).

Now you're probably thinking, that makes a very easy game to play. That's true, but - the suit offers you only limited time manipulation possibility, and the gameplay is challenging even with the special feature. What really makes this game extremely interesting is the (you guessed it!) multi-player game mode, where everyone has same capability. It's quite impressive (and for me, an average joe player) sometimes confusing, but a lot of fun!!

Last, but not least, thing worth mentioning is the graphics engine (called Saber3D) - it's visually stunning and attractive with really amazing textures of very high detail, especially on a hi-def TV set.

Be sure to check the game out, or at least download and play the demo before you buy. Really worth checking!