LocalEvalClient

class composer.utils.LocalEvalClient

Utility for creating a client for and invoking local evaluations.

invoke(payload)

Invoke a batch of provided payloads for code evaluations.

invoke_helper(payload)

Invoke a provided dictionary payload to a multiprocessing subprocess that performs code eval.

update_offline_helper(code_gen, test_input, test_output, entry_point, language, val)

Helper function to evaluate test case in a subprocess.

This function compiles the code generation, and runs the function from the entry point, before running the test input through the function and checking it against the test output.

Parameters

• code_gen (str) – The code generation to be evaluated.

• test_input (str) – The input of the test case

• test_output (str) – The output of the test case

• entry_point (str) – The name of the function to call

• language (str) – The language of the code generation

• val (multiprocessing.Value) – The value in which to save the final value of the test case