Processing HTML with Scala as if XML

The controversial XML API for Scala is still usefull for simple use cases. However it comes short when dealing with HTML as found on public websites since it is not well formed.




It is quite easy however to overcome this limitation by using the library tagsoup which allows to "fix" the HTML markup to make it look like XML.

You will add this dependency to your pom.xml :



This simple object can then be used to load an URL containing HTML markup as if it was XML.



Note that it also allows to set HTTP headers to the request, for instance if you want to use a cookie or a sessionId to get logged in.

Example calling code, which will list all HTML links from the loaded page :



And as a bonus, an object to force proxy settings from the code (for sure, you could also set it from the command line).

Easy Scala with Maven & Eclipse (scala 2.10)

I spent some time to figure this out ... not sold on sbt yet, and I am used to Eclipse 3.7.

So here is the simple procedure :

1. Install the Scala plugin for Eclipse from scala-ide.org. I use this update-site for scala 2.10
2. Install this eclipse plugin for maven / m2e : http://alchim31.free.fr/m2e-scala/update-site
3. You can then import easily a Maven project.
4. Right-click on the project, Configure > Add Scala Nature

Sample maven file :