Application failing to start

[Tracy51D]

Problem

We were contacted by a customer who was experiencing an issue whereby their application failed to start.

Investigation

From the error trace log received from the customer, the root cause is clearly indicated here:

Caused by: java.lang.Exception: The data was not in the correct format. Check the data file is uncompressed.

The library (device-detection.hash.engine.on-premise) is failing because the file provided is not in the correct format, most likely because:

• The file is still compressed (e.g., .zip, .gz), but the pipeline expects the raw/uncompressed .hash file.
• The wrong type of file is being used.
• The file path is misconfigured, and the pipeline is trying to load something invalid.

---

Troubleshooting Checklist

To help troubleshoot, could you please confirm the following:

• File path and access – Is the path to the data file correct, and does the engine have permission to read it?
• File format – Are you using a binary HashV41 file (.hash) or a CSV file? The engine requires a binary file, so please ensure you’re using one.
• Package version – Which version of the device-detection Maven package are you using?
• Pipeline configuration – Could you share your Pipeline configuration?
  • We’d like to try downloading the same data file and running an on-premise test using your setup.
  • This example may be particularly relevant: 51Degrees On-Premise Example
• Runtime environment – What environment is your application running in?
  • Please include OS version and Java version.
  • If you’re using a Docker image, sharing details would also help us reproduce the issue more accurately.

[Tracy51D]

• File path and access – Customer has confirmed the file path.
• File format – Customer is using the HashV41 file.
• Package version – Customer is using 4.2.8.
• Runtime environment – Customer is running in Docker using eclipse-temurin:17-jre-jammy. They believe this to be Ubuntu.
• Pipeline configuration – see below:

Pipeline Code

public Pipeline deviceDetectionPipeline() throws Exception {
    if (pipeline == null) {
        try {
            pipeline = new DeviceDetectionPipelineBuilder()
                .useOnPremise(externalConfig.getHashFileLocation(), false)
                .setPerformanceProfile(
                    fiftyone.pipeline.engines.Constants.PerformanceProfiles.LowMemory
                )
                .setDataUpdateLicenseKey(externalConfig.getLicenseKey())
                .build();
        } catch (Exception e) {
            throw new Exception(e.getMessage(), e);
        }
    }
    return pipeline;
}

[Tracy51D]

The issue was tested by our Engineers with a different version and no problems were observed. Both the new Licence Key and the old Licence Key generate the same HashV41.hash file, which confirms that the customer’s issue is specific to their environment.

In essence, the two files are identical, but switching between them causes the process to fail — or alternatively, the new file may be corrupted. If corruption is suspected, we recommend re-downloading the file.

For verification, the customer can compare their files using the resources available here.

We strongly recommend updating to the latest version of the library and re-downloading the file to ensure it is intact and not corrupted.

[douglac2]

We have most definitely downloaded the file with both keys. We have a configuration file which contains the license key. If I take the downloaded file and run the application with the OLD LICENSE KEY, we have no corruption. If I take the same file and specify the NEW LICENSE KEY in the configuration file and run it, we always have corruption after a period of time. At that point if I replace the file with a backup and put the OLD LICENSE KEY back in place, we have no corruption.

So that is the pattern we are seeing. No problem with the file until the NEW LICENSE KEY is introduced. It follows the key, not the file. We are investigating the code, but see no reason for this. We just want to be sure the OLD LICENSE KEY doesn't expire while we try to figure this out.

[douglac2]

One further point. The application doesn't fail to start unless it dies after the file has become corrupted. Before that it runs fine, but eventually becomes corrupted. That causes the app to fail, after which it won't start back up unless we replace the file with a good one.

Looked at the code again and don't see any issues with it so at a loss. The only use for the key is in the deviceDetectionPipeline method.

[Tracy51D]

@douglac2 - Thank you for sharing the private stack trace. Based on the error message, it appears that you are using a version prior to 4.4. We recommend upgrading to the latest version, which you can find here: https://central.sonatype.com/artifact/com.51degrees/device-detection/4.4.266.

The latest version introduces enhanced error messaging that includes the data file path, allowing you to verify the specific file in use and determine whether issues are caused by updates, file locks, incomplete writes, or uncompressed files.

Additionally, from your configuration we can see that you’re using the ‘low memory’ profile with temporary files disabled (parameter currently set to false). This configuration likely retains a lock on the source file, which can prevent in-place updates. To avoid file-lock issues, we recommend either:

• Enabling temporary files for on-premise deployments, or
• Switching to the in-memory (max performance) mode.

Regards
Tracy