Class InferenceOptions

java.lang.Object
com.codename1.ai.inference.InferenceOptions
public final class InferenceOptions extends Object

Configures how a reusable InferenceSession is created.

Accelerator requests are portable preferences rather than promises. With fallback enabled, a backend may execute on CPU when the requested delegate is unavailable. With fallback disabled, opening the session fails instead of silently changing the execution target. Android's NNAPI runtime can mix NPU and CPU operations without reporting full delegation, and the iOS Core ML delegate can schedule work across the Neural Engine, GPU, and CPU. Both mobile backends therefore reject InferenceOptions.Accelerator.NPU when fallback is disabled. iOS also rejects strict InferenceOptions.Accelerator.CORE_ML sessions because the delegate does not report whether unsupported model operations remained on LiteRT's CPU path.

  • Constructor Details

    • InferenceOptions

      public InferenceOptions()

  • Method Details

    • accelerator

      public InferenceOptions accelerator(InferenceOptions.Accelerator value)
      Requests an execution target for the model.
      Parameters:
      value - requested target; null restores InferenceOptions.Accelerator.AUTO
      Returns:
      this options object

    • threads

      public InferenceOptions threads(int value)
      Sets the CPU worker count. Non-positive values let the native runtime choose its default.
      Parameters:
      value - requested worker count
      Returns:
      this options object

    • allowFallback

      public InferenceOptions allowFallback(boolean value)

      Controls whether opening may fall back from an unavailable accelerator to CPU execution.

      On Android and iOS, setting this to false with InferenceOptions.Accelerator.NPU rejects session creation. On iOS it also rejects InferenceOptions.Accelerator.CORE_ML. Neither LiteRT's NNAPI delegate nor its Core ML delegate can prove that every operation ran on the requested accelerator instead of CPU or another processor.

      Parameters:
      value - true to permit CPU fallback
      Returns:
      this options object

    • getAccelerator

      public InferenceOptions.Accelerator getAccelerator()
      Returns:
      the requested accelerator, never null

    • getThreads

      public int getThreads()
      Returns:
      the requested CPU worker count, or a non-positive runtime default

    • isFallbackAllowed

      public boolean isFallbackAllowed()
      Returns:
      whether an unavailable accelerator may fall back to CPU