Skip to content

make Compiler thread-safe by moving CodeBuilder to instance scope#87

Open
yaodingyd wants to merge 2 commits intowbond:masterfrom
yaodingyd:thread-optimize
Open

make Compiler thread-safe by moving CodeBuilder to instance scope#87
yaodingyd wants to merge 2 commits intowbond:masterfrom
yaodingyd:thread-optimize

Conversation

@yaodingyd
Copy link
Copy Markdown

I'm using compilers in a supposedly thread-safe way:

def _get_compiler():
    compiler = getattr(_thread_local, "compiler", None)
    if compiler is None:
        compiler = Compiler()
        _thread_local.compiler = compiler
    return compiler

# then in subsequent code
_get_compiler().compile(my_text)

but running above code in a threadpool still running into issue

....
pybars/_compiler.py", line 629, in find_lookup
    self._result.grow([
    ^^^^^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'grow'

I think this is because each Compiler instance still reference to the same class field of _builder and _compiler, so their state get mutated cross thread

This change move these two fields to be instance property so when creating Compiler instance for each thread, each instance get isolated _build and _compiler

I don't have context on why we are keeping them this way so please correct me, appreciate more historical insights here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant