I had an issue today with the javac Ant task running out of memory when compiling a project (~800 files) under Java 6 on the Mac. It was fine under the default Java 5 compiler on the Mac, and it was fine under Java 6 on the PC. Initially, I set memoryMaximumSize on the javac task, which also required that fork be set.
This was OK, but since this was specific to compiling on the Mac, I figured the solution should be specific to my environment on the Mac. After looking at the source of the ant script, I realized that the ANT_OPTS variable could be set to contain an option (-Xmx512m) for the JRE running Ant, and that ANT_OPTS could be set from either ~/.antrc or ~/.ant/ant.conf. So, I created ~/.antrc with one line: ANT_OPTS=-DXmx512m.
(Update: oops, that should be ANT_OPTS=-Xmx512m)
And it worked perfectly, of course - so simple it had to work.
enjoy,
Charles.
Thursday, September 04, 2008
Thursday, August 28, 2008
Ant, JavaDoc, and CreateProcess error=87
I've been developing an Ant build file for a large-ish base of existing code. I've been using my Mac as the primary devleopment platform, but the ultimate target (for the other developers) is Windows. My task to build the javadocs runs fine on OS X, but the first time I rant it on Windows, I got the following:
c:\demo\build.xml:180: Javadoc failed: java.io.IOException: Cannot run program "c:\Program Files\Java\jdk1.6.0\bin\javadoc.exe": CreateProcess error=87, The parameter is incorrect
After about 10 minutes of Googling, I found a/the simple solution: add the useexternalfile attribute to the javadoc task in Ant. Voila , back in business, on the PC at least, and it didn't seem to break the build on the Mac.
Charles.
c:\demo\build.xml:180: Javadoc failed: java.io.IOException: Cannot run program "c:\Program Files\Java\jdk1.6.0\bin\javadoc.exe": CreateProcess error=87, The parameter is incorrect
After about 10 minutes of Googling, I found a/the simple solution: add the useexternalfile attribute to the javadoc task in Ant. Voila , back in business, on the PC at least, and it didn't seem to break the build on the Mac.
Charles.
Tuesday, August 12, 2008
Book: Essential SQLAlchemy
Essential SQLAlchemy by Rick Copeland
O'Reilly Media
ISBN 10: 0-596-51614-2 / ISBN 13: 9780596516147
This is a great book describing how to use SQLAlchemy to connect Python programs to databases. In fact, at the moment (mid-summer 2008), it is the book, since there are no other books on the subject, yet. Athough I am not (yet) a SQLAlchemy user, this book seems to cover all of the core topics in SQLAlchemy. The text includes many straightforward examples of how to use various facilities in SQLAlchemy and how to map various database programming problems into Python code via SQLAlchemy. Copeland also provides a whirlwind tour of some extensions to SQLAlchemy.
I heard about SQLAlchemy project on the This Week in Django podcast. Django doesn’t use SQLAlchmey, but it does use a similar object-relational mapper (ORM). As I mentioned, I haven’t used SQLAlchemy so I came into this book with a somewhat blank slate. I have, however, been programming in Python since before 1.0, and I’ve worked with database APIs and ORMs since the early 90s in C++, Java, and Python. So, I was familiar with the basic landscape of database programming, even if I hadn’t used SQLAlchemy. And, I’m currently working on a large Python project that is coded using the Python database API directly, which is very tedious. So, the whole time I was reading this book, I was looking at how to fit SQLAlchemy into this existing code base.
To be honest, the first chapter (the proverbial introduction) almost turned me off. The author starts out slowly enough, but then he starts touching on a huge number details, which were glazing my eyes over. However, the second chapter (getting started) started back at ground zero and stepped through everything in a nice clear fashion, and the rest of the book continued in that vein. He covers all the topics you would expect in a database programming book: queries, updates, joins, the built-in types, and how to hook in to provide support for your own types.
Something I didn’t realize about SQLAlchemy coming into this is that SQLAlchemy is both an ORM (what I expected) as well as a high-level, database-independent API. Which is to say, you can just access the database as tables, columns and rows rather than as classes, attributes, and object instances. Although I’d personally prefer to use the ORM, I can imagine cases where it might not be the right tool for the job, and it’s good to have a choice.
I was also surprised to see the ORM supports two styles of object-relational access: the data mapper pattern (which I had seen in Django and Hibernate) and the active record (used in Ruby). The author does a good job of explaining both of these and how to use them. He even devotes a whole chapter to Exlir, which is an extension that implements the active record pattern.
One thing that many people might consider odd is the fact that although SQLAlchemy is an ORM, the author waits until chapter eight to discuss how to map object inheritance hierarchies onto relational databases. Most books I’ve read on ORMs discuss this topic early, but I applaud Copeland’s decisions to hold off on discussing it. When books bring this up early (e.g., in chapter three), the discussion often gets bogged down in details, which glaze the reader’s eyes. I’ve dealt with the issue of inheritance mapping enough in ORMs I’ve used and those I’ve written enough that I’m not that interested in the topic (assuming the tool provides the typical, reasonable solutions), and was grateful that he held off on it.
One issue I had with the overall structure of the book is that I’m hard pressed to pigeon-hole the book. Books about a single technology such as SQLAlchemy usually occupy one end or the other of a spectrum. Either they’re hard-core references, often times copying-and-pasting API documentation from a web site (I really hate that), or they’re largish tutorials that may or may not contain enough technical meat. This is a short (roughly 200 page) book that contains plenty of technical meat, but it also includes some simple tutorial motivations for using various capabilities of the tool. Although this mix felt odd to me, I’m sure it will be perfect when I go to apply SQLAlchemy to my existing database projects since I don’t really want a hand-holding tutorial, but a pure reference wouldn’t quite work for me either when I’m just starting out.
In conclusion, Essential SQLAlchemy provides a thorough presentation of the SQLAlchemy tool for interfacing Python code to SQL databases. The author covers a number of different methods in which SQLAlchemy can be used to access databases from Python, and he provides plenty of details of the various APIs available to the programmer.
Enjoy,
Charles.
O'Reilly Media
ISBN 10: 0-596-51614-2 / ISBN 13: 9780596516147
This is a great book describing how to use SQLAlchemy to connect Python programs to databases. In fact, at the moment (mid-summer 2008), it is the book, since there are no other books on the subject, yet. Athough I am not (yet) a SQLAlchemy user, this book seems to cover all of the core topics in SQLAlchemy. The text includes many straightforward examples of how to use various facilities in SQLAlchemy and how to map various database programming problems into Python code via SQLAlchemy. Copeland also provides a whirlwind tour of some extensions to SQLAlchemy.
I heard about SQLAlchemy project on the This Week in Django podcast. Django doesn’t use SQLAlchmey, but it does use a similar object-relational mapper (ORM). As I mentioned, I haven’t used SQLAlchemy so I came into this book with a somewhat blank slate. I have, however, been programming in Python since before 1.0, and I’ve worked with database APIs and ORMs since the early 90s in C++, Java, and Python. So, I was familiar with the basic landscape of database programming, even if I hadn’t used SQLAlchemy. And, I’m currently working on a large Python project that is coded using the Python database API directly, which is very tedious. So, the whole time I was reading this book, I was looking at how to fit SQLAlchemy into this existing code base.
To be honest, the first chapter (the proverbial introduction) almost turned me off. The author starts out slowly enough, but then he starts touching on a huge number details, which were glazing my eyes over. However, the second chapter (getting started) started back at ground zero and stepped through everything in a nice clear fashion, and the rest of the book continued in that vein. He covers all the topics you would expect in a database programming book: queries, updates, joins, the built-in types, and how to hook in to provide support for your own types.
Something I didn’t realize about SQLAlchemy coming into this is that SQLAlchemy is both an ORM (what I expected) as well as a high-level, database-independent API. Which is to say, you can just access the database as tables, columns and rows rather than as classes, attributes, and object instances. Although I’d personally prefer to use the ORM, I can imagine cases where it might not be the right tool for the job, and it’s good to have a choice.
I was also surprised to see the ORM supports two styles of object-relational access: the data mapper pattern (which I had seen in Django and Hibernate) and the active record (used in Ruby). The author does a good job of explaining both of these and how to use them. He even devotes a whole chapter to Exlir, which is an extension that implements the active record pattern.
One thing that many people might consider odd is the fact that although SQLAlchemy is an ORM, the author waits until chapter eight to discuss how to map object inheritance hierarchies onto relational databases. Most books I’ve read on ORMs discuss this topic early, but I applaud Copeland’s decisions to hold off on discussing it. When books bring this up early (e.g., in chapter three), the discussion often gets bogged down in details, which glaze the reader’s eyes. I’ve dealt with the issue of inheritance mapping enough in ORMs I’ve used and those I’ve written enough that I’m not that interested in the topic (assuming the tool provides the typical, reasonable solutions), and was grateful that he held off on it.
One issue I had with the overall structure of the book is that I’m hard pressed to pigeon-hole the book. Books about a single technology such as SQLAlchemy usually occupy one end or the other of a spectrum. Either they’re hard-core references, often times copying-and-pasting API documentation from a web site (I really hate that), or they’re largish tutorials that may or may not contain enough technical meat. This is a short (roughly 200 page) book that contains plenty of technical meat, but it also includes some simple tutorial motivations for using various capabilities of the tool. Although this mix felt odd to me, I’m sure it will be perfect when I go to apply SQLAlchemy to my existing database projects since I don’t really want a hand-holding tutorial, but a pure reference wouldn’t quite work for me either when I’m just starting out.
In conclusion, Essential SQLAlchemy provides a thorough presentation of the SQLAlchemy tool for interfacing Python code to SQL databases. The author covers a number of different methods in which SQLAlchemy can be used to access databases from Python, and he provides plenty of details of the various APIs available to the programmer.
Enjoy,
Charles.
Sunday, July 06, 2008
Converting a Django Site to newforms-admin
Right after I learned about Django at OSCON 2006, I created a prototype to replace an existing PHP site. Due to various distractions, I haven't done much with that Django site since then and it never went live, but I've come back to it recently.
My first order of business was to upgrade from 0.95 to something modern. The first stop was to upgrade to trunk. Following the instructions on the Django site, I got it running on trunk and only had to change maxlength to max_length in my models.
Since, newforms-admin will be merging with trunk soon, porting to newforms-admin was my next step. I pointed my django install at my newforms admin svn tree and let it rip - i.e., I didn't bother reading any documentation or anything.
It blew up with a nasty-looking traceback that ended with this:
func(self, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'prepopulate_from'
A quick Google search turned up this thread on the Django Users mailing list. Five minutes of reading the newforms-admin wiki page and refactoring my code, and I was up and running. So, despite the ugly-looking traceback, it just wan't that bad.
Granted, my project is pretty small because I never had much time to work on it. However, all in all, the conversion process from 0.95 to the bleeding edge of newforms-admin was pretty painless.
Another happy Django user,
Charles.
My first order of business was to upgrade from 0.95 to something modern. The first stop was to upgrade to trunk. Following the instructions on the Django site, I got it running on trunk and only had to change maxlength to max_length in my models.
Since, newforms-admin will be merging with trunk soon, porting to newforms-admin was my next step. I pointed my django install at my newforms admin svn tree and let it rip - i.e., I didn't bother reading any documentation or anything.
It blew up with a nasty-looking traceback that ended with this:
func(self, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'prepopulate_from'
A quick Google search turned up this thread on the Django Users mailing list. Five minutes of reading the newforms-admin wiki page and refactoring my code, and I was up and running. So, despite the ugly-looking traceback, it just wan't that bad.
Granted, my project is pretty small because I never had much time to work on it. However, all in all, the conversion process from 0.95 to the bleeding edge of newforms-admin was pretty painless.
Another happy Django user,
Charles.
Book: Effective Java
Effective Java, 2nd Edition
by Joshua Bloch. ISBN 0321356683
This is the best book that I've read that I didn't know I needed to read. If you are a pretty good Java programmer, and you want to be better, this is a book you should definitely read.
Back in the day (BITD), we had Scott Meyer's book Effective C++
, and we needed it. Even without templates and all the "new" things in C++, C++ was a large and complex language. I loved Effective C++ not only because it had lots of tips to keep you out of trouble when coding C++, but also because the organization was brilliant: each tip was about the right length to be read while sitting on the can.
I ignored the first edition of Effective Java book when it came out because I smugly assumed that Java was so superior to C++ (Java is C++ with the pointy bits filed down) that it wasn't really necessary. And that may or may not have been true in the 1.0 or 1.1 days of Java, but with the release of Java 5, the language has certainly grown and is now sufficiently complex that a book like this is a necessity, especially if you've recently moved up to Java 5 or Java 6. Lucky for me, this book landed on my desk against my will.
The book uses that same sitting-on-the-can format from Effective C++; the information is broken down into nice, small bits of information that you can read one or two at a time. These are grouped into 10 sections. There are specific sections for Java 5 features like Generics and Enumerations, which is great for people like me who were stuck on 1.4 for far too long.
What is really amazing about this book is that I learned a lot about topics that I thought I already knew about. For example, I've used serialization in a number of formats (built-in and do-it-yourself) for years, but I realized there's a lot more that I had no idea about. I will certainly think twice before doing any serialization in the future.
Unless you're the sort that knows the Java Language Specification
by heart, you need to read this book.
Enjoy,
Charles.
by Joshua Bloch. ISBN 0321356683This is the best book that I've read that I didn't know I needed to read. If you are a pretty good Java programmer, and you want to be better, this is a book you should definitely read.
Back in the day (BITD), we had Scott Meyer's book Effective C++
, and we needed it. Even without templates and all the "new" things in C++, C++ was a large and complex language. I loved Effective C++ not only because it had lots of tips to keep you out of trouble when coding C++, but also because the organization was brilliant: each tip was about the right length to be read while sitting on the can.I ignored the first edition of Effective Java book when it came out because I smugly assumed that Java was so superior to C++ (Java is C++ with the pointy bits filed down) that it wasn't really necessary. And that may or may not have been true in the 1.0 or 1.1 days of Java, but with the release of Java 5, the language has certainly grown and is now sufficiently complex that a book like this is a necessity, especially if you've recently moved up to Java 5 or Java 6. Lucky for me, this book landed on my desk against my will.
The book uses that same sitting-on-the-can format from Effective C++; the information is broken down into nice, small bits of information that you can read one or two at a time. These are grouped into 10 sections. There are specific sections for Java 5 features like Generics and Enumerations, which is great for people like me who were stuck on 1.4 for far too long.
What is really amazing about this book is that I learned a lot about topics that I thought I already knew about. For example, I've used serialization in a number of formats (built-in and do-it-yourself) for years, but I realized there's a lot more that I had no idea about. I will certainly think twice before doing any serialization in the future.
Unless you're the sort that knows the Java Language Specification
by heart, you need to read this book.Enjoy,
Charles.
Tuesday, June 24, 2008
Configuring TortiseMerge for use with Mercurial
I've been using Mercurial primarily on a Mac, but also on a Windows box for a client. Since the Windows box isn't my primary platform, it tends to be under-configured/under-tricked-out. After my first conflicted hg merge, it was time to set up a merge program.
I guess Mercurial looks through the registry where it was finding some wacked out description for P4 merge. (The client currently uses P4.) It was referring to some path that didn't exist.
I have TortiseSVN on the machine from work for a different client, so I figured I'd use that merge program. Googling around, I didn't find an example, so here you go (even though it's not real hard - see MergeToolConfiguration for details):
[ui]
merge=TortoiseMerge
[merge-tools]
TortoiseMerge.executable=C:\Programs\TortoiseSVN\bin\TortoiseMerge.exe
TortoiseMerge.args=/mine:$local /theirs:$other /base:$base -o /merged:$output
enjoy,
Charles.
I guess Mercurial looks through the registry where it was finding some wacked out description for P4 merge. (The client currently uses P4.) It was referring to some path that didn't exist.
I have TortiseSVN on the machine from work for a different client, so I figured I'd use that merge program. Googling around, I didn't find an example, so here you go (even though it's not real hard - see MergeToolConfiguration for details):
[ui]
merge=TortoiseMerge
[merge-tools]
TortoiseMerge.executable=C:\Programs\TortoiseSVN\bin\TortoiseMerge.exe
TortoiseMerge.args=/mine:$local /theirs:$other /base:$base -o /merged:$output
enjoy,
Charles.
Tuesday, May 27, 2008
DVCS (Mercurial) for Students
I just had a bit of an epiphany for yet another reason why distributed version control systems (DVCS) like Mercurial rock. In an advanced software engineering class (e.g., a capstone project), it would be appropriate to have project teams using a SCM/VCS tool. At my campus, we've never pushed that because it can be a pain to set up a server for students to access. Gotta have a dedicated host for it. Firewalls have to be open. Permissions and users have to be set up. Yada, yada, yada.
However, with a DVCS tool like Mercurial, it would be trivial and wouldn't require any networking at all. Students could use the modern equivalent of SneakerNet: ThumbNet. Put the whole repo on a thumb drive, meet up with your project partners, push and pull, done. Or, if the project is small enough - just email whole repos around. In this context, "distributed" also means you don't need any support from The Man, and that's a good thing.
For better or worse, I don't teach those classes, but if I did....
Charles.
However, with a DVCS tool like Mercurial, it would be trivial and wouldn't require any networking at all. Students could use the modern equivalent of SneakerNet: ThumbNet. Put the whole repo on a thumb drive, meet up with your project partners, push and pull, done. Or, if the project is small enough - just email whole repos around. In this context, "distributed" also means you don't need any support from The Man, and that's a good thing.
For better or worse, I don't teach those classes, but if I did....
Charles.
Friday, May 16, 2008
@Override is your friend
When Java 5 came out, I had my head down teaching and wasn't really paying attention. I've since started working with it and have been using annotations for "big" tasks like Hibernate/JPA metadata. I was pretty underwhelmed by @Override - one of the only stock annotations. When I implement toString, I know I'm overriding the method on Object. Who cares?
The other day I was beating my head into the desk trying to figure out why a Swing table wasn't editable. I had overridden the isCellEditable method on JTable, but the cells weren't editable. Then, I remembered something from the annoations tutorial I'd read at some point: "While it's not required ... it helps to prevent errors." So, I added @Override, and sure enough - I'd misspelled the method name, just the sort of error that @Override can prevent.
I've got the religion. And like any recent convert, I suggest you get it too.
enjoy,
Charles.
The other day I was beating my head into the desk trying to figure out why a Swing table wasn't editable. I had overridden the isCellEditable method on JTable, but the cells weren't editable. Then, I remembered something from the annoations tutorial I'd read at some point: "While it's not required ... it helps to prevent errors." So, I added @Override, and sure enough - I'd misspelled the method name, just the sort of error that @Override can prevent.
I've got the religion. And like any recent convert, I suggest you get it too.
enjoy,
Charles.
Monday, May 12, 2008
Data Driven, my eye!
I've started using the WebTest web testing framework. Mostly, it's pretty cool. However, I have a bone to pick with screencast demonstrating the dataDriven task.
Charles - aka Cranky Pants.
Update:
OK, so it sucked to be me, but not any more. I figured out my various issues with the dataDriven task. It turns out that the screencast (clearly) shows them developing in the tests directory of a WebTest project. I missed that and tried to use dataDriven in the top-level build.xml in a target that didn't declare wt.defineTasks as a dependency. Guess what wt.defineTask does - yup, it does the taskdef.
I coupled that breakthrough with breaking down and using Excel to create the xls file (instead of exporting from Google) and viola - I'm livin' large and no longer cranky.
enjoy,
Charles.
P.S. Apologies to Cheech and Chong.
- There's a "slide" that says "Do you know the dataDriven Ant task?" I know of no such standard Ant task. It turns out that it's specific to WebTest. Not really clear.
- They show no configuration steps to use it, implying that it works out-of-the-box. I don't know if my environment is wacked (I installed from the developer build, as they suggested), but I had to add an Ant taskdef referring to com.canoo.ant.task.PropertyTableTask, and I only found that by looking in the source.
- The screencast shows running ant at the command line, which is how I've been running my tests, but I had to run their webtest script instead. Again, maybe my installation is wacked.
- I really wish it could handle data in a TSV/CSV/text file, since I don't have Excel installed on the machine where I'm running these tests, but it only seems to accept an xls file.
- Just to add insult to injury, Google Spreadsheet (which I'm using to generate the data file) seems to append a bunch of empty lines to my spreadsheet, which causes the dataDriven task to repeat the last line 90-odd times.
Charles - aka Cranky Pants.
Update:
OK, so it sucked to be me, but not any more. I figured out my various issues with the dataDriven task. It turns out that the screencast (clearly) shows them developing in the tests directory of a WebTest project. I missed that and tried to use dataDriven in the top-level build.xml in a target that didn't declare wt.defineTasks as a dependency. Guess what wt.defineTask does - yup, it does the taskdef.
I coupled that breakthrough with breaking down and using Excel to create the xls file (instead of exporting from Google) and viola - I'm livin' large and no longer cranky.
enjoy,
Charles.
P.S. Apologies to Cheech and Chong.
Wednesday, May 07, 2008
Universal???
Apple has released Java6 for the Mac, which I have been eagerly awaiting. However, their download page is a bit contradictory:
This is limited to 64-bit Intel macs (which is fine with me), but yet they still call it Universal. What's Universal about that? The only way I could imagine it being less universal is if it they specified a number of cores or CPU speed.
Anyway, Java6 is good stuff for those of us on Universal Core2 Duo iMacs.
Charles.
This is limited to 64-bit Intel macs (which is fine with me), but yet they still call it Universal. What's Universal about that? The only way I could imagine it being less universal is if it they specified a number of cores or CPU speed.
Anyway, Java6 is good stuff for those of us on Universal Core2 Duo iMacs.
Charles.
Subscribe to:
Posts (Atom)
