class composer.utils.LocalEvalClient[source]#

Utility for creating a client for and invoking local evaluations.


Invoke a batch of provided payloads for code evaluations.


Invoke a provided dictionary payload to a multiprocessing subprocess that performs code eval.

update_offline_helper(code_gen, test_input, test_output, entry_point, language, val)[source]#

Helper function to evaluate test case in a subprocess.

This function compiles the code generation, and runs the function from the entry point, before running the test input through the function and checking it against the test output.

  • code_gen (str) โ€“ The code generation to be evaluated.

  • test_input (str) โ€“ The input of the test case

  • test_output (str) โ€“ The output of the test case

  • entry_point (str) โ€“ The name of the function to call

  • language (str) โ€“ The language of the code generation

  • val (multiprocessing.Value) โ€“ The value in which to save the final value of the test case