Nim's Cynical Pleasantries Code, swords and a dirty mind!

8Feb/150

Getting started with JAX-RS & Gradle

Managed to waste an hour of my life today, trying to set up a JAX-RS + Gradle project. Gradle wasn't really the problem, nor was JAX-RS, but putting it all together in a way that works took longer than expected.

So without further ado, I present to you a simple quickstart project you can use to kickstart JAX-RS + Gradle projects.

The Gradle build file is pretty minimalistic. There's no need for web.xml. And I've included an example resource.

build.gradle:

description = 'JAX-RS Quickstart'
group = 'org.lick.me.jaxrs'
version = '1.0-SNAPSHOT'
apply plugin: 'java'
apply plugin: 'eclipse'
sourceCompatibility = 1.8
 
repositories {
        mavenCentral()
}
 
 
buildscript {
        repositories {
                jcenter()
        }
        dependencies {
                // Embedded Jetty 9 Gradle Plugin
                classpath (group: 'com.sahlbach.gradle', name: 'gradle-jetty-eclipse-plugin', version: '1.9.+')
        }
}
apply plugin: 'jettyEclipse'
apply plugin: 'war'
 
dependencies {
        // JAX-RS 2.0(.1)
        compile 'javax.ws.rs:javax.ws.rs-api:2.0.1'
 
        // Jersey 2.15
        compile 'org.glassfish.jersey.containers:jersey-container-servlet:2.15'
 
        // Servlet 3
        compile 'javax.servlet:javax.servlet-api:3.1.0'
}

Extra quick quickstart:

git clone git@github.com:Nimlhug/jax-rs-quickstart.git
cd jax-rs-quickstart
gradle jettyEclipseRun
Tagged as: , No Comments
29Dec/141

I guess I should be honoured?

Someone seems to have named their programming language after me. Thanks for the compliment ;-)

Filed under: Uncategorized 1 Comment
27Dec/140

More Open Source donations

The end of the year is upon us, and the kind folks over at GnuPG (gpg) are asking for donations. I pitched in along with many others, and I would encourage anyone who values online privacy to do the same. GPG is a critical part of open source development, extremely useful for signing releases or securing communications.

If you have a little cash to spare, consider making a donation to a FOSS project you like or use. A little goes a long way. Thanks!

Tagged as: , No Comments
4Nov/140

Gradle gotcha: duplicate project names

Assume the following project structure:

Product
├──build.gradle
├──settings.gradle
├──FeatureA
│   ├── Foo
│   │   ├── build.gradle
│   ├── Bar
│   │   ├── build.gradle
├──FeatureB
│   ├── Foo
│   │   ├── build.gradle
│   ├── Quf
│   │   ├── build.gradle

I don't think this is a terribly unreasonable structure: we have a product with a bunch of features, and each feature is split into a couple of modules. This works just fine in gradle. As long as FeatureB/Foo doesn't have a dependency on FeatureA/Foo. Gradle doesn't seem to be terribly fond of multi-module projects. At least not when you have a deeper, nested structure.

As soon as FeatureB/Foo has a dependency on FeatureA/Foo, gradle goes belly up. It won't tell you, though. It'll just quietly carry on. It won't resolve the dependency, and you'll end up with compile errors.

FeatureB/Foo/build.gradle:

dependencies {
	compile project(':FeatureA:Foo')
}

The reason is actually quite simple. Gradle uses the directory name as the project name. Instead of looking at the whole path in the tree, which would be a lot more sensible, it just looks at the last leaf in the tree. So we have two projects named Foo, and gradle chokes. Not giving an error/warning strikes me as a bug, though, but still. Not terribly unreasonable.

Thankfully, there's a fix. It involves renaming your projects via settings.gradle in the root project:

settings.gradle:

// List our projects/subprojects
include ':FeatureA:Foo'
include ':FeatureA:Bar'
include ':FeatureB:Foo'
include ':FeatureB:Quf'
 
// Rename each child project to its path: Foo becomes FeatureA-Foo or FeatureB-Foo, depending on the Foo.
rootProject.children.each { p ->
        p.children.each { module ->
                module.name = p.name + '-' + module.name
        }
}

Now you can add a project dependency and things will Just Work™:

dependencies {
	compile project(':FeatureA:FeatureA-Foo')
}

All fixed. Much better.

Tagged as: No Comments
17Aug/140

Dear PHP Developers, please stop using the closing tag ?>

Dear PHP Developers, please stop using ?> at the end of your scripts. There's no need for it, it's ugly, and it's a nightmare when you've got whitespace hiding behind the tag.

So stop, please. Just, stop.

Tagged as: , No Comments
20Jul/142

Laptops

My Thinkpad x230 gave me a bit of a scare recently. The display died out of the blue, and Lenovo's support was an absolute nightmare. In spite of next business day on site warranty, it took them nearly two weeks to actually fix the bloody thing. The first time the tech showed up, he replaced the broken panel with another broken panel. Not too useful, that.

While I was in limbo, unsure if it would ever get fixed under warranty, and wondering how much it would cost me to have it fixed, I spent some time researching alternatives.

Sadly, I found none. It seems like literally no one makes laptops for people who use use them anymore. There's loads of consumer-grade crap out there, but nothing useful. Especially in the 12.5" range.

There are a couple of recurring failures.

Keyboard & Mouse

x240

The Thinkpad x240 -- "new and improved" -- comes with a trackpad that takes up more space than the fucking keyboard. The top buttons (used in conjunction with the nipple) are part of the trackpad. Apparently if you disable the trackpad, you're out of luck and out of buttons. Great. Turning off the trackpad, of course, is mandatory, given that the thing is so fucking huge that my thumbs constantly touch it inadvertently. Especially annoying when focus follows pointer.

The picture doesn't do it justice, but another problem with the x240 is that the function keys got smaller and are now closer to the number row than before. Makes for an overall less enjoyable experience.

207647

dellXPS13-9333_3

These ASUS & Dell pieces of shite both seem to be missing a bunch of keys (page up, page down, home, insert, end, ...) and seem to have completely flat keys. Apparently there's a market for people who don't type.

Mechanical failures waiting to happen

xps

asus

Whoever designed these amazing ultra portable laptops seems to have forgotten that people will be moving around with them. Those display look like they'll snap in half the first time you bump into anything. And seriously, who the fuck needs a detachable display? Who buys this? What do they do for a living? Not software engineers, that's for sure.

Touchscreen

Apparently everybody and their grandmother wants a touchscreen. I can't for the life of my imagine why. Having to use a mouse (due to terrible window management and non-keyboard-friendly websites) is already a waste of time. Having to touch my screen (and leaving greasy fingerprints everywhere) is even worse. Fuck that shit.

So ...

Laptops that don't suffer from any of the above flaws are few and far between, and usually have a CPU that's slower than my phone's, or less memory than my toaster. Anything that ships with an AMD CPU isn't even worthy of being a coaster, either. Makes you wonder how they're still relevant.

Apparently there are literally no laptops on the market for software engineers. There's all kinds of crap designed around this Windows 8 insanity with its useless tablet-like interface. I'm sure these machines are perfectly fine for consuming information like some kind of brain dead zombie, but there's no universe in which these machines are useful for creating or getting any work done.

Needless to say, I'm quite happy that Lenovo did repair my x230 in the end. Hopefully it'll outlive its warranty. With a bit of luck, the x250 or x260 won't suck quite as much...

Filed under: Uncategorized 2 Comments
2Jul/140

Fun with Gradle Plugins – Integration Tests

Currently in the process of migrating a 10000 line Ant build to Gradle. Not quite as fun as it sounds, but at least the Gradle build should be faster, more maintainable and hopefully free from cruft.

One of this build's peculiarities is that it executes Unit Tests and Integration Tests at different points in the build cycle. The Integration tests rely on nasty things like databases, IBM MQ and LDAP. Things that are difficult to mock out & slow your build down to a grind if you execute these tests too frequently. Some of them are really System tests, if you're being pedantic, but let's just ignore that. The point is, there's a category of tests that we don't want to see executed during the normal unit test execution.

Full code available on GitHub.

Selim Öber explains how to accomplish this on his blog. I wanted to see if I could turn it into a plugin instead, as it will have to be applied to many different projects (made more difficult by a heterogenous coding environment. Not everything is a Java project).

The Goal

The goal of this plugin is to be able to execute tests in src/integration/java by executing the integrationTest target. These tests will expressly not be executed during the normal build cycle. These are to be run manually (or by your CI engine or whatever) periodically, as opposed to continuously on each build.

buildSrc

The buildSrc folder is Gradle's magic box. It lives in your project root and contains -- as the name might imply -- some of your build's source code. Gradle is smart enough to pick up on the stuff there, and will automagically compile it when it changes.

I chose groovy for this plugin, so I'll just let Gradle know about that, and I'll politely ask it to make the Gradle API available for me to use.

buildSrc/build.gradle

apply plugin: 'groovy'
 
repositories {
	mavenCentral()
}
 
dependencies {
	compile gradleApi()
}

Next is the actual plugin.

buildSrc/src/main/groovy/org/lick/me/gradle/IntegrationTestPlugin.groovy

package org.lick.me.gradle
 
import org.gradle.api.Plugin
 
import org.gradle.api.Project
import org.gradle.api.tasks.testing.Test
 
/**
 * Applying this plugin will let gradle know that the project contains integration tests.
 * These can be executed at a later point in time than unit tests -- when a database etc 
 * become available. 
 */
class IntegrationTestPlugin implements Plugin<Project> {
	@Override
	void apply(final Project project) {
 
		project.sourceSets {
			integTest {
				java.srcDir project.file('src/integration/java')
				resources.srcDir project.file('src/integration/resources')
			}
		}
 
		project.dependencies {
			integTestCompile project.sourceSets.main.output
			integTestCompile project.configurations.testCompile
			integTestCompile project.sourceSets.test.output
			integTestRuntime project.configurations.testRuntime
		}
 
		project.task('integrationTest', type: Test, description: 'Runs the integration tests.', group: 'Verification') {
			testClassesDir = project.sourceSets.integTest.output.classesDir
			classpath = project.sourceSets.integTest.runtimeClasspath
		}
 
		project.task('allTests', dependsOn: [project.test, project.integrationTest], description: 'Runs all tests.', group: 'Verification') {
 
		}
	}
}

Now all we need is a bit of magic to let Gradle know that we want to be able apply this plugin in other projects.

buildSrc/src/main/resources/META-INF/gradle-plugins/integrationTests.properties

implementation-class=org.lick.me.gradle.IntegrationTestPlugin

Now we can add the plugin to your projects.

apply plugin: 'integrationTests'

Running gradle tasks will now result in two extra tasks in the Verification section.

tasks

Filed under: Uncategorized No Comments
28Aug/137

Yet Another Battery Widget (Awesome 3.5.1)

Yet another battery widget for Awesome. This one actually works (shock! horror!) on Awesome 3.5.1 on my Thinkpad x230. Your mileage may vary. Colours used are from the excellent Solarized colour scheme. Behold the mighty widget, in all its unobtrusive glory!

battery

The implementation is in two parts: a simple shell script to output the battery status, and a bit of rc.lua tweaks to display the widget. This is mostly the result of a bit of copy/pasting from different sources I forgot to bookmark. Oh well.

~/bin/battery.sh:

#!/bin/bash
 
healthy='#859900'
low='#b58900'
discharge='#dc322f'
 
capacity=`cat /sys/class/power_supply/BAT0/capacity`
if (($capacity <= 25));
then
        capacityColour=$low
else
        capacityColour=$healthy
fi
 
status=`cat /sys/class/power_supply/BAT0/status`
 
if [[ "$status" = "Discharging" ]]
then
        statusColour=$discharge
        status="▼"
else
        statusColour=$healthy
        status="▲"
fi
 
echo "<span color=\"$capacityColour\">$capacity%</span> <span color=\"$statusColour\">$status</span>"

Add the following snippets to /path/to/awesome/rc.lua. I'll attempt to indicate the approximate location at the top of each snippet.

Create the widget..and don't forget to adjust the path to the battery.sh script.

-- This goes below the line containing mytextclock = awful.widget.textclock()
 
-- Create a battery widget
battery = wibox.widget.textbox()
function getBatteryStatus()
   local fd= io.popen("/path/to/battery.sh")
   local status = fd:read()
   fd:close()
   return status
end

Add the widget..

-- This goes above the line containing right_layout:add(mytextclock)
    right_layout:add(battery)

Get the widget to refresh every 30 seconds. Put this somewhere near the end of the config file.

-- Battery status timer
batteryTimer = timer({timeout = 30})
batteryTimer:connect_signal("timeout", function()
  battery:set_markup(getBatteryStatus())
end)
batteryTimer:start()
battery:set_markup(getBatteryStatus())

That's all! Restart awesome and you'll see a relatively purdy yet unobstrusive battery status display.

Tagged as: 7 Comments
3Aug/139

Java Date Performance Subtleties

A recent profling session pointed out that some of our processing threads were blocking on java.util.Date construction. This is troubling, because it's something we do many thousands of times per second, and blocked threads are pretty bad!

A bit of digging led me to TimeZone.getDefault(). This, for some insanely fucked up reason, makes a synchronized call to TimeZone.getDefaultInAppContext(). The call is synchronized because it attempts to load the default time zone from the sun.awt.AppContext. What. The. Fuck. I don't know what people were smoking when they wrote this, but I hope they enjoyed it ...

Unfortunately, Date doesn't have a constructor which takes a TimeZone argument, so it always calls getDefault() instead.

I decided to run some microbenchmarks. I benchmarked four different ways of creating Dates:

// date-short:
    new Date();
//date-long: 
    new Date(year, month, date, hrs, min, sec);
// calendar:
    Calendar cal = Calendar.getInstance(TimeZone);
    cal.set(year, month, date, hourOfDay, minute, second)
    cal.getTime();
// cached-cleared-calendar:
//    Same as calendar, but with Calendar.getInstance() outside of the loop, 
//    and a cal.clear() call in the loop.

I tested single threaded performance, where 1M Dates were created using each method in a single thread. Then multi-threaded with 4 threads, each thread creating 250k Dates. In other words: both methods ended up creating the same number of Dates.

Lower is beter.

Click to enlarge. Lower is beter.

With exception of date-long, all methods speed up by a factor of 2 when multi-threaded. (The machine only has 2 physical cores). The date-long method actually slows down when multi-threaded. This is because of lock contention in the synchronized TimeZone acquisition.

The JavaDoc for Date suggests replacing the date-long call by a calendar call. Performance-wise, this is not a very good suggestion: its single-threaded performance is twice as bad as that of Date unless you reuse the same Calendar instance. Even multi-threaded it's outperformed by date-long. This is simply not acceptable.

Fortunately, the cached-cleared-calendar option performs very well. You could easily store a ThreadLocal reference to an instance of a Calendar and clear it whenever you need to use it.

More important than the raw duration of the Date creation, is the synchronization overhead. Every time a thread has to wait to enter a synchronized block, it could end up being rescheduled or swapped out. This reduces the predictability of performance. Keeping synchronization down to a minimum (or zero, in this case) increases predictability and liveness of the application in general.

Before anyone mentions it: yes, I'm aware that the long Date constructors are deprecated. Unfortunately, they are what Joda uses when converting to Java Dates. I've proposed a patch, but while doing a bit more research for this blog post, I've come to the conclusion that my patch needs a bit of refining as it is still too slow (though it no longer blocks). In the mean while, I hope that the -kind?- folks at Oracle will reconsider their shoddy implementation.

I've also heard rumours that Joda will somehow, magically, replace java.util.Date in JDK 8. Not sure how that's going to work with backwards compatibility. I'd be much happier if java.util.Date would stop sucking quite as much. And if SimpleDateFormat were made thread-safe. And ... the list goes on.

28Apr/130

Character sets, time zones and hashes

Character sets, time zones and password hashes are pretty much the bane of my life. Whenever something breaks in a particularly spectacular fashion, you can be sure that one of those three is, in some way, responsible. Apparently the average software developer Just Doesn't Get It™. Granted, they are pretty complex topics. I'm not expecting anyone to care about the difference between ISO-8859-15 and ISO-8859-1, know about UTC's subtleties or be able to implement SHA-1 using a ball of twine.

What I do expect, is for sensible folk to follow these very simple guidelines. They will make your (and everyone else's) life substantially easier.

Use UTF-8..

Always. No exceptions. Configure your text editors to default to UTF-8. Make sure everyone on your team does the same. And while you're at it, configure the editor to use UNIX-style line-endings (newline, without useless carriage returns).

..or not

Make sure you document the cases where you can't use UTF-8. Write down and remember which encoding you are using, and why. Remember that iconv is your friend.

Store dates with time zone information

Always. No exceptions. A date/time is entirely meaningless unless you know which time zone it's in. Store the time zone. If you're using some kind of retarded age-old RDBMS which doesn't support date/time fields with TZ data, then you can either store your dates as a string, or store the TZ in an extra column. I repeat: a date is meaningless without a time zone.

While I'm on the subject: store dates in a format described by ISO 8601, ending with a Z to designate UTC (Zulu). No fancy pansy nonsense with the first 3 letters of the English name of the month. All you need is ISO 8601.

Bonus tip: always store dates in UTC. Make the conversion to the user time zone only when presenting times to a user.

Don't rely on platform defaults

You want your code to be cross-platform, right? So don't rely on platform defaults. Be explicit about which time zone/encoding/language/.. you're using or expecting.

Use bcrypt

Don't try to roll your own password hashing mechanism. It'll suck and it'll be broken beyond repair. Instead, use bcrypt or PBKDF2. They're designed to be slow, which will make brute-force attacks less likely to be successful. Implementations are available for most sensible programming environments.

If you have some kind of roll-your-own fetish, then at least use an HMAC.

Problem be gone

Keeping these simple guidelines in mind will prevent entire ranges of bugs from being introduced into your code base. Total cost of implementation: zilch. Benefit: fewer headdesk incidents.