JVM-Sandbox module causes HBase ImportTsv tool to fail

Author: YLChen-007Created Dec 14, 2024Updated Dec 30, 2024

Problem Description

When using a custom JVM-Sandbox module, the HBase ImportTsv tool fails to execute and throws the following error:

Error: Could not find or load main class [main]

The issue disappears when the -javaagent:/home/hbase/.opt/sandbox/lib/sandbox-agent.jar option is removed from the JAVA_TOOL_OPTIONS.

Although my module does not explicitly enhance or interfere with any target classes or methods, the issue occurs as long as the module is loaded.


Steps to Reproduce

  1. Configure JVM-Sandbox:

    • Use sandbox-agent.jar as the Java agent.
    • Enable the custom module patch-point-monitor.
  2. Run the HBase ImportTsv tool, for example:

    bash
    hbase org.apache.hadoop.hbase.mapreduce.ImportTsv ...
  3. Add the following environment variable before running the tool:

    bash
    export JAVA_TOOL_OPTIONS="-javaagent:/home/hbase/.opt/sandbox/lib/sandbox-agent.jar"
  4. Execute the command, which results in the following error:

    Error: Could not find or load main class [main]
  5. Remove the JAVA_TOOL_OPTIONS environment variable, and the ImportTsv tool works as expected.


Module Code

Below is the code for my JVM-Sandbox module:

java
package org.sandbox.module.cyl;

import com.alibaba.jvm.sandbox.api.Information;
import com.alibaba.jvm.sandbox.api.LoadCompleted;
import com.alibaba.jvm.sandbox.api.Module;
import com.alibaba.jvm.sandbox.api.ModuleLifecycle;
import com.alibaba.jvm.sandbox.api.listener.ext.AdviceListener;
import com.alibaba.jvm.sandbox.api.listener.ext.EventWatchBuilder;
import com.alibaba.jvm.sandbox.api.resource.ModuleEventWatcher;
import org.kohsuke.MetaInfServices;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.annotation.Resource;

@MetaInfServices(Module.class)
@Information(id = "patch-point-monitor", version = "0.0.1", author = "[email protected]")
public class MonitorPatchPointModule implements Module, LoadCompleted, ModuleLifecycle {
    final Logger logger = LoggerFactory.getLogger("MonitorPatchPointModule-LOGGER");

    @Resource
    private ModuleEventWatcher moduleEventWatcher;

    @Override
    public void loadCompleted() {
        logger.info("MonitorPatchPointModule loadCompleted ** ");
        // No actual enhancements are applied
        logger.info("MonitorPatchPointModule loadCompleted finish ** ");
    }

    @Override
    public void onLoad() throws Throwable {
        logger.info("MonitorPatchPointModule onLoad ** ");
        logger.info("MonitorPatchPointModule onLoad finish ** ");
    }

    @Override
    public void onUnload() throws Throwable {
    }

    @Override
    public void onActive() throws Throwable {
    }

    @Override
    public void onFrozen() throws Throwable {
    }
}

Environment Information

  • JVM-Sandbox Version: 1.3.0 (assumed version, replace with your actual version)
  • HBase Version: 2.4.15
  • Hadoop Version: 3.3.6
  • JDK Version: 1.8.0_422
  • Operating System: CentOS 7

Troubleshooting Steps

  1. Load JVM-Sandbox without the module
    When only the sandbox-agent.jar is loaded (without any modules), the ImportTsv tool works correctly.

  2. Disable parts of the module logic
    Commenting out all code related to ModuleEventWatcher in the module does not resolve the issue.

  3. Check module dependencies
    Verified that the module has no conflicting dependencies with HBase or Hadoop, but the issue persists.


Possible Causes

  1. Classloader Conflict
    The AgentClassLoader introduced by JVM-Sandbox may interfere with HBase's or Hadoop's classloading mechanism.

  2. Implicit Effects of Module Lifecycle
    Even without explicit enhancements, the module's lifecycle methods (e.g., onLoad, loadCompleted) may trigger some framework-level logic that affects HBase's execution.

  3. HBase Tool Dependency
    The ImportTsv tool relies on Hadoop's classloading mechanism, which might be disrupted by JVM-Sandbox's instrumentation.


Expected Behavior

The JVM-Sandbox module should not interfere with the execution of HBase tools like ImportTsv. Alternatively, there should be a way to isolate the module's impact on the application classloader.


Suggestions

If possible, please help confirm the following:

  1. Does JVM-Sandbox provide a mechanism to avoid impacting specific classloaders (e.g., HBase's classloader)?
  2. Is there a way to enable more detailed debugging logs to pinpoint the root cause of the issue?
  3. Are there recommended practices for isolating module classloaders to prevent conflicts with the target application's classloader?