java.lang.IllegalArgumentException: unresolved address at java.net.DatagramPacket.setSocketAddress

[levtomer66]

Hi, at totally random times I get this exception in one of my production machines:

ERROR com.codahale.metrics.ScheduledReporter - Exception thrown from StatsDReporter#report. Exception was suppressed. java.lang.IllegalArgumentException: unresolved address at java.net.DatagramPacket.setSocketAddress(DatagramPacket.java:316) at java.net.DatagramPacket.<init>(DatagramPacket.java:144) at java.net.DatagramPacket.<init>(DatagramPacket.java:178) at com.readytalk.metrics.DatagramSocketFactory.createPacket(DatagramSocketFactory.java:30) at com.readytalk.metrics.StatsD.send(StatsD.java:98) at com.readytalk.metrics.StatsDReporter.reportGauge(StatsDReporter.java:252) at com.readytalk.metrics.StatsDReporter.report(StatsDReporter.java:175) at com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) at com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

I captured with tcpdump the DNS packets to see where is the failure comes from, and I found a small lag from the dns server with resolving the name of the statsd server, I replaced the configuration to IP address instead of hostname and once again this error occured, this time I could see that it's trying to reach AWS hostname of the statsd server for the ip:

x.x.x.x

it is

ip-x-x-x-x.eu-west-1.compute.internal
It's important to mention I'm running on AWS EC2 m5 type machines and using Route53 private-zones as DNS server.
no idea how it infers this IP, any thoughts how can I suppress this error ?

1. Can add retries to the dns server in order to not throw after first try and retry the dns name resolution?
2. Can I disable the name resolve and communicate via IP only ?