Top.Mail.Ru
July 20th, 2004 - Java developers — LiveJournal
? ?

Java developers

July 20th, 2004
Image

03:13 pm - Imagespecialagentm - JMS experience?

Anyone out there used the Java Messaging System? I have a project where I need to coordinate several systems. One system is accumulating data and storing it for archival purposes. As the archival system writes information to disk, it needs to alert another system that will read back the data to do some analysis on it.

I'm thinking of using JMS for this purpose -- send a message to a controller box that says "hey, new data archived for server foo.com". Then the analysis boxes can consume those messages to start various analysis tasks on the data.

Anyone have experience in something like this? I mean, just the message-passing part (I have a handle on the rest). I'm assuming JMS is the right approach, but I'm open to other suggestions.

Some data points:

I expect a very large number of messages being generated. There are 2000 hosts feeding me data, and the data gets cached to disk every 5-10 minutes, with a pretty even spread over that time. So maybe on the order of 100-200 messages per minute being generated?

I expect to have 4-5 boxes looking for incoming messages; essentially each "worker" box is just waiting for a command to go do a data analysis, then it returns to a wait state afterwards.

This all has to run on both Windows and Linux. Performance on Windows doesn't matter, but that's where we do development so I need it to at least sort of work enough for testing.
Image

05:59 pm - Imagetrajano - Unit testing

This is taken from my reply to an artcile on theserverside.com regarding unit testing

One of the things that Mike Spille states in his article was that anyone can write a better test framework in a day. It may be true, but when you are concentrating on developing your application, you shouldn't be wasting your time building another framework and use most of your time developing your own application.

Its basically the same concept as using the jakarta commons libraries, most of the things in there we can do ourselves, but the question would be why bother re-writing the piece of code over and over again. And if you try to justify saying that you'd want your own private library with little dependencies, you have to ensure that you maintain those small pieces of code over and over again rather than taking advantage of what is already out there.

Using JUnit is quite simplistic I have to agree. But would you rather test your small routine with the entire application all at once or just write a simple test case. Sure you can just create a main() method or separate class to test it out, but writing a JUnit test is similiar to writing the main() method anyway only there is some IDE and tool support for the tool so you can take advantage of the reports it would generate.

Assertions in JUnit actually prevent you from having to look at your test run once you decide what the correct output is supposed to be. I cheat and sometimes use printlns to display the result and copy and paste it into the assertions, because the result is sometimes too tedious to type.

JUnit being lightweight is not a hinderance, its a bonus for it. Being lightweight allows it to be easily embedded into other tools such as JPerf, Eclipse, Ant and Maven. Its also easy to prove that it just works because it does so little.

Of course I wouldn't use JUnit alone when I do integration level testing (that's just nuts at times since its going to make my test case longer than it has to be). I'd use extensions to JUnit such as HttpUnit, StrutsTestCase, JFCUnit and Cactus. Tools like HttpUnit (and a bit of refactored code to reduce the code base I maintain) allow me to translate Use Cases to test code to verify that things flow as expected (of course that would take a bit more experience to do since it is more complicated). And integration level tools allow me to regression test cheaply.

I am not as ridiculous and run all unit tests all the time on my development PC, that's just nuts. I wait till the nightly builds execute and run the test cases for me and let the build manager assign blame appropriately.

As another person said on the thread, unit tests also aid in refactoring. I usually some deadlines are hard and I just write whatever to get the thing working. The code I write may not be good and/or it might just be a 1000 line class, but it works. I don't write my JUnit test cases then just make sure that the demo works. Once I do get that working, I just add the unit test case afterwards, something minimal where I just prove that a boundary value works properly. Then I go on to another component. Usually, I will come back to that piece of code later, the most common reason being, I have to cut and paste come code from it, at which point I do a bit of refactoring to reduce the code base (I want to reduce the active code base as much as possible, because its harder for new people to pick up if the code base is too big).

And to rebut the other thread where someone said that unit tests take 2/3s the time to make. I've been on application maintenence group, most developers who haven't gone through that probably just write code to get the customer happy and pay, application maintenance teams have to make sure the bloody thing runs everytime and figure out why something goes wrong. Working on application maintenance does build up sufficient skills, wisdom and maturity to find ways of avoiding code problems before it gets to them.

Automated unit tests are a useful tool when you have teams who have to make sure that the code works because it reduces the costs for them to fix any bugs that the development teams make, by reducing regression costs. Of course like any part in application development, they have to be done properly.

To finish this reply, here are some rules which I personally have trouble myself from doing when I write test cases. (in order of difficulty)


  1. make your tests throw exceptions (don't bother catching them unless you need to)

  2. write at least one sentence on what you are testing

  3. refactor your tests, a lot of tests can get common setup and teardown routines

  4. put unit tests reports as part of your nightly build cycle.

  5. turn use cases into test scripts

  6. build data that you are testing on, don't rely on data to be there.

  7. remove data that you are finished testing with if possible.

  8. make things testable without the container



x-posted in Imagetrajano
 

06:55 pm - Imageex_juan_gan - Order in XML

There is a lot of controversy regarding how and why one can order items in and XML file.

Generally speaking, setting items order in XML should not differ from ordering them in a relational database or in any other representation form. But due to XML readability, certain misconceptions penetrate the minds of those who want to a) keep information regarding items order in XML, and b) keep it simple.

First, several definitions.

If X is a set, we define partial order on set X as a binary relationship R that is reflective, transitive and antisymmetric, that is,

Axiom a) xRx;
Axiom b) xRy and yRz => xRz;
Axiom c) xRy and yRx => x = y.

See http://en.wikipedia.org/wiki/Partial_order for reference.

Total order, or linear order is a partial order where all elements are comparable (see http://en.wikipedia.org/wiki/Totally_ordered_set):

Axiom d) xRy or yRx.

We encounter partial order when we need to specify the order of actions - some of them have to follow each other, some of them can be executed in parallel. In programming languages there is no easy way to specify parallelism; operators are written sequentially and are perceived as following each other (although actually they may be shuffled when executed).

It is natural to place totally-ordered items sequentially - e.g. in XML, if items specify certain actions, and if item B lexicographically follows item A, action B is perceived as following action A. Linear order, though, is not what one expects to use when describing processes. That's why, for instance, in Ant one has to specify target dependencies in depends attribute, for example:

<target name="clean" depends="init, cleanReport">
</target>

Note that the order in which init and cleanReport are listed in depends does not matter: this is a set of targets on which target named clean is dependent. This is how the actual order is specified, not by placing clean lexicographically after init and cleanReport. The order here is partial order, it is not linear, and cannot be expressed by position of an item in the file.

In a sense, it is easier when we deal with relational databases - we know that these do not have any predefined order at all, so that to specify order we just have to introduce an additional table that would store this binary relationship.

It is curious that dependencies in Ant build.xml are specified in an attribute. This does not look like an attribute; more, to produce the list, one has to parse the value of the attribute, which seems to me an action more intimate than necessary. Why not have a dependency subelement, and list all dependencies there, something like this:

<target name="clean">
  <depends>
    <target name="init"/>
    <target name="cleanReport"/>
  <depends>
</target>

Looks very natural - but unfortunately we usually cannot do this, and all thanks to the peculiar features of SAX, the API that some people use for retrieving information from XML sources. When you use SAX, what you get in your handler is the element name (in our case it is target); there is no information regarding the context in which it is used. So, in SAX we won't be able to tell the difference between the item <target name="init"/> within dependency element and outside, where this target is actually defined.

Interesting. The only other case when you cannot use a name outside its declaration is only in shell scripts, where you declare a variable, and then have to prepend its name with $, dollar sign, every time you use it.

(This weakness of SAX, inability to determine element's context, is the cause of introducing otherwise feature that otherwise would be totally useless: namespaces. While namespaces would not help in the situation like the one described above, they help to find out what kind of element is the one that we are parsing if the document contains elements of different origin that happen to have the same name. Notice that, if the parser know the context, there is hardly any need to use namespaces, since it is always obvious what it is about.)

So, is there any real life case when the physical, lexicographic order of elements in XML would bear any meaning? I don't know. Do you? Please let me know if you do.
 

11:51 pm - Imagegamahucheur - Freeware Utility


NumericsDisplay is a freeware tool spun off from the forthcoming PraxioLogic Class File Editor. NumericsDisplay is conceptualized primarily as a utility for converting floating point numbers from Java expression to IEEE 754-1985 and vice versa, or integers from Java expression to decimal and octal and hexadecimal and vice versa; but it also offers the functionality of an RPN programmer's calculator.

screen shotCollapse )

The program itself is a pure Java .jar. It is freeware and not adware or spyware. It distributed both in a tar-ball and in a Windows self-extracting archive.
Powered by LiveJournal.com
Image