Iterating over the lines of a file in Java

If you’re trying to write a standard command-line tool in Java you are going to experience pain.

In Python, if we want to do something with each line on input to your program, we can do something like:

import sys
for ln in sys.stdin:
    print(ln)  # (We have an extra newline but who's counting?)

In Java, after some fighting, the two closest alternatives I can find are:

import java.util.Scanner;
class IterateInputScanner {
    public static void main(String[] args) {
        for(String ln : (Iterable)(() -> new Scanner("\n"))) {

or, less fashionably, and even more horribly:

class IterateInputReader {
    public static void main(String[] args) throws {
        BufferedReader reader =
            new BufferedReader(new InputStreamReader(;
        String ln;
        while((ln = reader.readLine()) != null) {

Note that if you are reading from a stream other than stdin you will probably want to wrap this in a try-with-resources to close the stream when you’ve finished.

Take your pick, and I hope it saves you some fighting.

Setting up a sane Maven project

Today I have spent hours trying to get wrangle Maven into letting me set up a sane Java project. The hardest parts were enforcing warnings-as-errors, and getting Maven to shut up a bit.

Some code that warns

My first problem was writing some Java code that causes the compiler to emit warnings. For some reason I can’t understand, my Java 1.8 compiler was not emitting warnings (even with -Xlint:all) for unused variables, use of deprecated methods, or unknown @SuppressWarnings types (suggestions from SO 1752607).

Instead, I had to use an unnecessary cast:

$ cat src/tast/java/
public class ExampleTest {
    public void warn() {
        String fixmePlease = (String)"Hello";

Now, finally, I got a warning:

$ javac -Xlint:all src/test/
src/test/ warning: [cast] redundant cast to String
        String s = (String) "Hello!";
1 warning

Maven compiler settings for warnings-as-errors

I tried a vast set of combinations of properties like maven.compiler.failOnWarning and maven.compiler.fork (as specified in the maven compiler plugin docs) before giving up on properties. Making a property called maven.compiler.failOnWarning seems to have no effect whatsoever, ever.

So I decided I must try the (very verbose) plugin tag containing a configuration tag, as also specified in the maven compiler plugin docs. After a lot of messing about with flags that seemed incompatible, and Maven silently ignoring things it didn’t understand, I came to a working config.

On the way, I discovered that setting the “fork” property to true is a non-starter, because Maven simply drops the compiler output in that case, meaning you can’t see what is going wrong when it does.

Finally, I had a pom file like this:

cat pom.xml
<?xml version="1.0"?>
<project xmlns="" xmlns:xsi="" xsi:schemaLocation="">

Which manages successfully to tell the compiler to show all warnings, and to fail when it sees one. (Thanks to SO 9192613, among others.)

I dare not change anything, for fear that it will stop working again without any explanation.

Quieting Maven

If you tell Maven to be quiet with -q or -e it will merrily fail the build because of a warning, but not tell you what the warning was.

Maven does not appear to have a command line option to set the log level to display warnings and errors only, but you can force it to do so by setting the environment variable MAVEN_OPTS like this:

MAVEN_OPTS=MAVEN_OPTS=-Dorg.slf4j.simpleLogger.defaultLogLevel=warn mvn clean test-compile

(Thanks to SO 4782089.)

And, with some guesswork (partly based on Configuring Maven) I found that if I put something similar in .mvn/jvm.config I didn’t have to type it every time:

$ cat .mvn/jvm.config 

Failing on warnings, and seeing them!

I don’t know whether to feel triumphant or defeated, but, it works!

$ mvn clean test-compile 
[WARNING] src/test/java/[3,20] redundant cast to java.lang.String
[ERROR] src/test/java/ warnings found and -Werror specified
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:testCompile (default-testCompile) on project unmatcheddrrepair: Compilation failure
[ERROR] src/test/java/ warnings found and -Werror specified
[ERROR] -> [Help 1]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]

I wish I could suppress double-reporting of the error, and the extra rubbish at the end (and the super-long absolute paths of each file that push the actual errors off the side of the screen), but at this point, I must try and do what I was trying to do in the first place.

Guess how much I am liking Maven at this point.

Simple example of Netty 4 usage

I feel the title of this post over-promises, since I was not able to make an example that seemed simple to me.

Anyway, here is a near-minimal example of how to use Netty to make a server that shouts back at you whatever you say:

import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.util.CharsetUtil;
import io.netty.util.ReferenceCountUtil;
import java.nio.charset.Charset;

class NettyExample
    public static void main( String[] args ) throws Exception
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
            new ServerBootstrap()
                .group( bossGroup, workerGroup )
                .channel( NioServerSocketChannel.class )
                .childHandler( new Init() )
                .bind( 1337 ).sync().channel().closeFuture().sync();

    private static class Init extends ChannelInitializer
        public void
        initChannel( SocketChannel ch ) throws Exception
            ch.pipeline().addLast( new ShoutyHandler() );

    private static class ShoutyHandler extends ChannelInboundHandlerAdapter
        public void channelRead( ChannelHandlerContext ctx, Object msg )
                Charset utf8 = CharsetUtil.UTF_8;
                String in = ( (ByteBuf)msg ).toString( utf8 );
                String out = in.toUpperCase(); // Shout!
                ctx.writeAndFlush( Unpooled.copiedBuffer( out, utf8 ) );
                ReferenceCountUtil.release( msg );

        public void exceptionCaught(
            ChannelHandlerContext ctx, Throwable cause )

The lines that actually do something useful are highlighted in red. If anyone knows how to make it shorter, please comment below. It seems a lot to me.

To run this, do:

sudo apt-get install openjdk-8-jdk
wget ' -O netty-all-4.1.5.Final.jar'
javac -Werror -cp netty-all-4.1.5.Final.jar:. && java -cp netty-all-4.1.5.Final.jar:. NettyExample

Then in another terminal:

echo "Hello, world" | nc localhost 1337

and observe the response:


Comparison with Node.js

Just for comparison, here is an approximate equivalent in Node.js:


var net = require('net');

var server = net.createServer(
    function( socket ) {
            function( data ) {
                socket.end( data.toUpperCase() );

server.listen( 1337, "localhost" );

To run it, do:

sudo apt-get install nodejs-legacy
node shouty.js

Then in another terminal:

echo "Hello, world" | nc localhost 1337

and observe the response:


Ambiguous names in Java due to non-normalised unicode – but all OK in Python

In Java and several other languages, identifiers (e.g. method names) are allowed to contain unicode characters.

Unfortunately, some combinations of unicode characters are logically identical. For example, á (one character: Latin Small Letter a with Acute U+00E1) is the same as á (two characters: Latin Small Letter A U+0061, and Non-spacing Acute Accent U+0301). These combinations are not just similar – they are identical by definition.

Java does not do any normalisation on your code before compiling it, so two identifiers containing equivalent but different unicode combinations are considered different (ref: JLS 7 section 3.8).

$ cat 
public class U {
    static String \u00e1() { return "A WITH ACUTE"; }
    static String a\u0301() { return "A + NON-SPACING ACUTE"; }
    public static void main(String[] a) {
$ javac && java U

We can define and use two functions called á and á and they are totally independent entities.

But don’t do this.

Python 3 also allows unicode characters in identifiers, but it avoids the above problem by normalising them (ref: Python 3 Reference, section 2.3):

$ cat 
#!/usr/bin/env python3

def á():
    print("A WITH ACUTE")

def á():
    print("A + NON-SPACING ACUTE")


$ hexdump -C 
23 21 2f 75 73 72 2f 62  69 6e 2f 65 6e 76 20 70  |#!/usr/bin/env p|
79 74 68 6f 6e 33 0a 0a  64 65 66 20 c3 a1 28 29  |ython3..def ..()|
3a 0a 20 20 20 20 70 72  69 6e 74 28 22 41 20 57  |:.    print("A W|
49 54 48 20 41 43 55 54  45 22 29 0a 0a 64 65 66  |ITH ACUTE")..def|
20 61 cc 81 28 29 3a 0a  20 20 20 20 70 72 69 6e  | a..():.    prin|
74 28 22 41 20 2b 20 4e  4f 4e 2d 53 50 41 43 49  |t("A + NON-SPACI|
4e 47 20 41 43 55 54 45  22 29 0a 0a c3 a1 28 29  |NG ACUTE")....()|
0a 61 cc 81 28 29 0a 0a                           |.a..()..|
$ ./ 


The second definition overwrites the first because they are considered identical. You can call it via either way of saying its name.

Both ways of working are scary, but I’d definitely choose the Python 3 way if I had to.

How to analyse a .phd heap dump from an IBM JVM

If you have been handed a .phd file which is a dump of the heap of an IBM Java virtual machine, you can analyse it using the Eclipse Memory Analyzer Tool (MAT), but you must install the IBM Monitoring and Diagnostic Tools first.

Download MAT from I suggest the Standalone version.

Unzip it and run the MemoryAnalyzer executable inside the zip. Add an argument to control how much memory it gets e.g. to give it 4GB:

./MemoryAnalyzer -vmargs -Xmx4g

Once it’s started, go to Help -> Install new software.

Next to “Work with” paste in the URL for the IBM Developer Toolkit update site:

Click Add…

Type in a name like “IBM Monitoring and Diagnostic Tools” and click OK.

In the list below, an item should appear called IBM Monitoring and Diagnostic Tools. Tick the box next to it, click Next, and follow the wizard to accept the license agreements and install the toolkit.

Restart Eclipse when prompted.

Choose File -> Open Heap Dump and choose your .phd file. It should open in MAT and allow you to figure out who is using all that memory.