legacy databases with datamapper

For my current project I choose datamapper wich has some advantages over ActiveRecord. I need a thread safe ORM which doesnt need to do much but performing real fast. I am dealing with legacy table names but couldnt find any documentation how this is done though its very simple:

Your model needs a method storage_name like :

def self.storage_name(repository)
  "your_legacy_table_name"
end

Repository can be ignored if you are just dealing with one database.

ps: afcool83 mentioned a smarter way to do this:

storage_names[:default]='your_legacy_table_name'


the power of testing

I really thought we are over this point yet but it turns out to be a mistake. I work in a couple of projects in parallel so I meet a lot of developers and see many different types. I am really surprised to realize that the need to write tests still is so much underestimated.

All the classics like "no time to write tests", "is it really worth the effort" to "I thought this is just a webapp, so there is no need to write tests".

It happend to often too me, so here is my lesson learned if you start a new project with existing code. Ask for the tests. Take a peek a their test code, it gives you far more insight then any arbitrary code. It doesnt mean dont take any projects where no tests have been written, but it gives you different arguments. Its like dealing with legacy code, you cannot start implementing features as everything is fine.
But if the project leaders dont realize this fact forget the project, its a death march.


To say it with Obie Fernandez words
"Delivering Rails applications without automated test coverage is irresponsible and unprofessional."

620 Errors while testing with Google Geocoding

If you are using Googles Geocoding Api you might experience occasional problems with 620 errors, especially while testing. The reason for this is Google prohibits too many calls in a short period. But for most testcases geocoding is not an issue so you should disable it for these.

Fortunatly in Ruby its easy to override existing methods to mock them. I use the geokit plugin and with it this could be done by the following method:

def disable_geocoding
    GeoKit::Geocoders::Geocoder.instance_eval do
    	def geocode(a)
    		res = GeoKit::GeoLoc.new
    		res.lat = 13.423007
    		res.lng = 52.534194
    		res.success = true
    		res
    	end
    end
  end

Include this in your testhelper and call disable_geocoding in your setup method.

Annoying floating point conversion in Ruby

Try (4.85*100).to_i in irb and you will get:
484

This is weired and totaly unexpected for me. I understand that floating points are not accurate and conversions have their limits. Type "%.32f" % 4.85 and you see the real value of this float. The reason is that to_i does just a truncate.

If you want the expected result you have to use:
(4.85*100).round.

Some GTD criticism

I believe in chaos. Not that I say I love it. We are all in it by nature. For me its not a fight against, its more a dealing with it. You can try to keep its entropy low but you always need creativity to deal with the unforseen. GTD on the other hand is very strict and makes many assumptions. For me its hard to put all my tasks into one big pot and to decide when to write this or that task down for instance. GTD is sometimes too inflexible for certain situations. Sometimes projects need more attention, and they have much shorter cycles than less imortant or long running tasks. They grow bigger and they split or subprojects emerge and so on.
I am sure I did not get everything Allen is saying us so I am misinterpreting one or the other point. But I mean, hey, this book is like a bible, and I dont like the idea you have to read the same book many times to get a simple thing.
Dont get me wrong, I am still using GTD, not too dogmatic, as its fits and I am sattisfied with it. Good tools like Omnifocus are a great help there. I am still experimenting with it but I think dealing with the habits is more important because they are the source when we did a failure. So for me GTD is just a small tool in a complex world ;-)

Render a simple Javascript with Rails

Though its a simple task there is no well documented solution. Most I googled did not work. I just needed to write a large javascript array dynamicly. If you just do a render :text => js your resulting content type will be text/html which was a problem for the IE. But setting the mime_type like

render :text => js,:mime_type => Mime::Type.lookup("application/javascript")

does not do the trick. What you have to do is setting the content type in the headers variable:

headers["Content-Type"] = "text/javascript".

I got the best results when I also added a .js to the route so you can call it like a static javascriptfile. If you want to do more complicated things you may check out the minus_r plugin of Dan Webb:

Fat model

Today I had the chance to enjoy a dinner with David Black and some other Ruby folks in Berlin. We were discussing about legacy Rails code. Many newbies tend to write fat controllers with many pages of codes and 30 something methods. One was arguing that this is also Java style. I am now thinking thats not prefered even in Java though it was engouraged in J2EE because Entity Beans had been such a crap. Its just functional programming style where you end when you dont understand object orientation.

state of the art javascript gui

That gui of the north280 guys is really awesome. you have got to check out their presentation tool. everything is so smooth, it doesnt look like a webgui. at least with my Safari. If you want to read about the background story and the developers tuaw has more. hopefully they publish their javascript libraray objective-j soon.

Merb presentation

While I heard a lot of postive stuff about merb I had no time to play with it, but this presentation on confreaks is definitly worth seeing. Ezra Zygmuntowicz explains some of the advantages of merb. I like the idea of a hacker should understand his framework he is working with. Therefor no magic in the framework. The magic is for your code. What I found interesting as a Rails developer he also has some concrete critics at Rails, mainly at the Actionpack where he started refactoring some really old parts there.

Upgrade to rails2

Yeah, since last week palabea runs on rails2 on production. The actual refactoring was painless and even the production deploy had just minor bugs. There are tiny changes coming with the upgrade - like empty parameters are now blank and not nil - which fell through our test coverage. So a quick look over all your controllers is needed if you want to avoid surprises in production.

Adding parameters to routes

Routes are definitly one of the black holes in rails development. Most of the time you are dealing with standard configurations. You have to learn them, but there are not too many options you have.
But last time I wanted something special: to add an parameter to a url if a special route is called. We have /videos and we have /videos?lecture=true. Both are videos, share the same table and model, but they are treated differently in the GUI. So I thought definining a new route /lectures would be nice. I should call the same methods as for /videos but with a special parameter added. There is no way to do it. But hey I thought thats Ruby. What about adding a block, or override a special method. Then I started digging into the code. ... Wow, thats hot stuff, where to start? Jamis Buck tried but I couldn't figure out where and how to place my hook.
Finally I resignated and added a before_filter to the controller which does a regexp check against the request_uri. This does also the job, but I added code to the controller which is part of the routes logic, so really I dont like the solution. But it works fine for now.

External rails plugin resources (and how to get rid of them)

Since it seems state of the art to keep plugins as external resources I tried myself to install them via script/plugin -x URL. 

Maybe I am missing some points here, of course it has some advantages, but in reality this has some big flaws. I would like to keep control over my sourcecode and dont want to deploy on production suddenly realizing there is a new version of a plugin out there. 

These external biests are really hard to handle with svn, like merging. Or imagine you need to make changes to these resouces.
Sad but true I found no documentation how to remove these plugins. It cost me some effort to find out since I am no svn guru. But its pretty easy. Just call

svn propedit svn:externals vendor/plugins. 

If you get a message that no SVN_EDITOR is defined you have to do that:

export SVN_EDITOR=mate

Delete all the external resource entries (file should be blank afterwards) and commit the vendor/plugins directory. Maybe you need a clean checkout before to do this properly.

Smartfolders in Netnewswire

I am a long time user of NetNewsWire, the best newsreader you can find. Its sometimes hard to manage the huge and ever growing mass of blogs, news etc, so I tried a new strategy: I started using smart folders for the mass and try to keep the number of blogs I read regularly low. So I can concentrate on the things I am really interested in.

Nice presentation by Hohpe

Infoq posted a nice presentation by Gregor Hohpe, where he talks about SOA. For him domain specific languages could be an answer to over complicated general languages like BPEL.

Also if you havent done so read his article about starbucks and the two phase commit: a very nice analogy about asynchronous communication in a real world example.

Use Burlap to build a simple Java-Ruby bridge

Yes, I know, there is this great tool JRuby which will hopefully be an essential part of the JDK in the near future, and yes there is REST where Restlet seems to be the most promising framework, but all I wanted is to speak with a very simple Java Interface from a Rails application. So I thought there must be an easier way, but it has to deal with ongoing changes to the interface. The answer for me was the Burlap protocol which you get with Spring for free. All you have to do is to expose your Java bean with the Burlap exporter:

 <bean id="fooService" 
  class="com.agelion.server.impl.DummyService">

 <bean name="/FooService" 
   class="org.springframework.remoting.caucho.BurlapServiceExporter">
     <property name="service" ref="fooService"/>
     <property name="serviceInterface" 
        value="com.agelion.server.FooBackendService"/>
 </bean>
 

. Thats it for the Java side. Burlap is a very simple protocol where you dont map classes, what you get is a map with key, value pairs. But thats fine for me.

But also the Ruby side is easy to implement. I am still a newby with Ruby but that was relativly quick done (though its still not as easy to get information like you get for J2EE for instance, where you get too much information).
Because the Burlap protocol just supports the POST method you need the low level Ruby methods for Http:

Net::HTTP.start('localhost', 8080) do |request| 
    response = request.post('/backendservice/FooService', 
     '<burlap:call><method>myMethod</method></burlap:call>')
 

And here we go. All you have to do is to parse this XML:

 vals = []
 XPath.each( Document.new(response.body),
     "//burlap:reply/list/map/*") 
 {|p|  vals << p.text }
 

And as I said, what you get are key-value pairs with the exception of the first element which descripes the original type (classname).
I converted it to a map skipping the first element in the array:

 @bashvals = {}
 1.step(vals.length-1, 2) 
    { |i|  @bashvals[vals[i]] = vals[i+1] }

Writing tests that test failure

Often forgotten because most of the time you want to proove that your code runs. Yes I know we all tend to be lazy. But to be complete in a sense of Sir Carl Popper you also have to falsify your assumption. You are forced to think about deeper about what your code does and what it should not do. But even further you proove that exceptions are treated correct in case of an error or worse like it happened to me it gets swallowed. I was about to refactor an ancient piece of software written in the early days of Java development. I wrote a test for a new feature and was happy to see my test was successfull. But writing a failure test I recognized that every exception was swallowed by an nice try {} catch (Exception) block just because this test failed.

Testing with maven2

While I like maven2 because of its productivity benefits now a simple task was taking me some time of research. I have to deal with some legacy code where some tests exists, but these are plain java code with main methods. I look forward to refactor them to junit tests, but step by step. So I have to tell maven what class to use and what not. For two reasons I'd like to gather them all in a testsuite. First I could start the suite easily from my IDE, and second I have another testsuite, which will require an external Corbaserver running. So I would like to seperate these tests.

There are two points to consider:

1) Because maven doesnt seem to work with suites you have to write a TestCase like this:


public class SuiteTest extends TestCase {

TestResult tr = null;

public void testSuite() {
TestSuite suite = (TestSuite) YourTestSuite.suite();
suite.run(tr);
}

public void run(TestResult res) {
tr = res;
testSuite();
}
}


2) Whereas in maven1 it was simple to configure the unit tests with the <unittest> tag, this doesnt work in maven2 anymore. In the pom.xml you have to configure the surefire plugin which runs the test. Thats not obvious for a maven1 user, so its not easy to find the right documentation. But the maven2 concept is to be more plugable. So its easier to include Testng for instance.

Amateur projects

It happend to me again. Today I found myself in a new project where everything seems to go wrong. The climate is non productive, they have sitting 12 developers, where I guess 2 or 3 good ones can do the same job. They have absolutly no unit tests, no build server, no documatation, neither external nor within code, no code reviews, a lot of changing developers, an over complicated build system which doesnt work, too much code which is solved by open source libraries, and so on. If you work like this you get used to it, but if you know there is a much different way there is only one way: you have to tell. The decision is as always between leave this as soon as possible or try to change, which of course is much harder. But this depends on the situation, as long as there is enough money nobody cares. Its easier to convice them if they realize its a case of surving.

Checkout Maven

I worked for a company where they used Maven as a standard build tool. They used maven1, but I had some spare time the last couple of weeks, so I used it to play with maven2. I read the book Maven from the Developers Notebook series and Better Build with Maven.

First of all, why use Maven instead of Ant, which are most familiar with?

1) Its a collection of best practices of build & configuration managment: Where do I put my configuration files, how do I integrate common tools like xdoclet, clover or checkstyle, make project reports, where do I put my external jars, how to announce a new version, do the deployment and so on. When I checkout old projects of mine, which are based on Ant, I always have to look in the build.xml how and where files are copied and so on. With maven thats not the case, because everything is just at the right place. And because with Ant there is no standard build structure, every project is different.

2) Reuse: Maven has a plugin concept. Every task is a plugin which gets loaded when you use it the first time, or gets updated if there is a new version. Even common tasks like clean are plugins. So you dont need to write scripts, you just call mvn clean without having any definition of clean in your project file.

3) Productivity: you start creating a new project with mvn archetype:create -DgroupId=your.main.package -DartifactId=projectname and you get a basic project structure. You define your final archive like jar, war or ear, add your external jar dependencies to pom.xml, call mvn eclipse:eclipse or mvn idea:idea, you get your project files for your IDE and can start working within seconds. And finally mvn install compiles your code, runs your unit tests and packages your archive. All that is done without the need to write any build code. Last but not least maven2 is faster than Ant.

There is much more you get with maven for free, like cruisecontrol integration, repository integration plus reports and stats, you can run your httpunit or cactus tests out-of-the-box.

The dependency managment is great: Maven supports transitive dependencies, so if you use Struts you just add Struts to your dependencies and get all jars needed by Struts. Maven provides automatic conflict managment. And every dependency has a scope, some jars are just needed at compile time, for tests and so on. All that is nothing you have to worry about, you just add the scope tag to your dependency.

So I really believe every java developer should consider using Maven, at least give it a try and play around with it a little bit.

Fun with testing

I will not talk about the need for unit tests. This I would say is common sense for a modern thinking developer. I recently had to do some small changes in a project. The main developer had already left the company and I got a brief introduction. They all felt very secure because there was a testcase for every single usecase, and also for every possible error. But these were integrationtests, they wrote something into a database, sent requests per URL and parsed the result with httpunit.
So far so good, everyone was happy, but me. Why that? Of course because there were no unit tests at all. It took me about three days to get the tests running. They used real services instead of mocks, so the tests ran just on a dedicated machine, because of firewall restrictions. The logs where analysed to ensure proper behaviour, but because they where running on an external machine, there where some shell scripts to pipe the logs to a socket. I played around for a while, even tried to use logj4 SocketAppender but failed to get the whole thing running. God knows why. So I dropped that validation. The script to run the tests swallowed every exception. The tests were running about 30 minutes, so after reading the latest news you just saw that some of the tests failed but had no glue why.

That all pissed me of, so I started writing unit tests and mock objects. One of my first tests failed and I was wondering why. I discovered a bug where two flags from an external database had been read in the wrong order. You have to know these where not just two flags nobody cared. These had been two of five values these app was evaluating, one of the reasons why these service was written. To evaluate these and to redirect the user depending on the combination of these.

WTF?! But what about the integration tests? At least they ran without an error. So after looking at the testcode, I couldnt believe my eyes: the good old boy was reversing his error and exchanged the flags at initialization time, so everything was looking fine again. I can imagine the situation, maybe he was wondering why his tests failed and thought, oh i might have exchanged the flags. So after reversing them he got the green bar.

This app was in production state for about one year and everyone was happy, but me .....