We need to upgrade all systems to TRAPI 2.0 in dev

Although we will continue to maintain TRAPI 1.6 support for a little while longer, we now need to implement TRAPI 2.0 in ARAX and the ARAX GUI.

@bazarkua

The requirements.txt in commit 5975cec doesn't work from a fresh pip install -r requirements.txt, because the catrax-xcrg package (based on the current pinned commit) pulls a catrax-xcrg pyproject.toml file that specifies a version for opentelemetry-api, 1.17.0, that is inconsistent with (and much older than) the pinned dependency (1.45.0) for opentelemetry-api that is specified in the latest ARAX requirements.txt. Noting it here because, I think, we will need to fix this before attempting to auto-deploy to cicd.rtx.ai via the fantastic ARAX auto-deploy system developed by @bazarkua . @edeutsch showed me how to hack around this to get the fastapi server working in the meantime, so this is not blocking work on fork_mode.

@edeutsch

To get the trapi2.0 branch working locally on your development machine, in addition to following the steps in the ARAX Maintenance SOP document, for now, you will need to do the following updated steps:

in requirements.txt, comment out fastapi and uvicorn, and change the opentelemetry-api to 1.17.0
in a fresh python3.12 virtualenv, install the requirements.txt using pip install -r requirements.txt
install fastapi using pip install --upgrade fastapi
install uvicorn using pip install --upgrade uvicorn
when you run the fastapi application server, need to use your web browser to open
RTX/code/UI/interactive/index.html and then click on "Settings' and change the
ARAX_QUERY_URL string to: http://localhost:5001/devED/api/arax/v2.0/query.
change flask_config.json to app_server_config.json (put in RTX/code/UI/OpenAPI/python-flask-server/openapi_server); as of commit 722f0f3

h/t to @edeutsch for the pip install --upgrade workaround.

So, I have ARAX working locally on my MBP, without fork_mode (getting same error seen on arax.ncats.io/devED):

Remove old Flasks tests never really used #2904

I have done the renaming of flask_config.json to app_server_config.json, locally. Pushing the commit shortly.

Remove endpoints migrated to 2.0. Fix #2904

found a combination that seems to work. Fix #2904

Skip Jaeger for the moment. Fix #2904

#2904 merge master into trapi2.0

first thing: pytest dont even start on my MBP locally on trapi2.0, conftest fails on import

env: fresh python 3.12 venv, pip install -r requirements.txt from trapi2.0 (scipy bumped to 1.16.3 locally bc 1.15.3 wheel dont load on macOS 27, not related to this error)

$ python --version
Python 3.12.11

$ cd code/ARAX/test
$ python -m pytest --collect-only -q -p no:cacheprovider
ImportError while loading conftest '/Users/bazarkua/Work/RTX/code/ARAX/test/conftest.py'.
conftest.py:11: in <module>
    from kp_info_cacher import KPInfoCacher
../ARAXQuery/Expand/kp_info_cacher.py:20: in <module>
    from ARAX_response import ARAXResponse
../ARAXQuery/ARAX_response.py:8: in <module>
    from openapi_server.models.log_entry import LogEntry
E   ModuleNotFoundError: No module named 'openapi_server'

same error when importing only this module:

$ cd ../ARAXQuery
$ python -c 'import ARAX_response'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/bazarkua/Work/RTX/code/ARAX/ARAXQuery/ARAX_response.py", line 8, in <module>
    from openapi_server.models.log_entry import LogEntry
ModuleNotFoundError: No module named 'openapi_server'

reason: LogEntry import was added in 6799b2c

but ARAX_response.py dont add python-flask-server/ to sys.path before it:

RTX/code/ARAX/ARAXQuery/ARAX_response.py

Lines 5 to 8 in 744e574

    
           import datetime 
        
           import os 
        
           from openapi_server.models.log_entry import LogEntry

like ARAX_messenger.py does:

RTX/code/ARAX/ARAXQuery/ARAX_messenger.py

Lines 23 to 24 in 744e574

    
           sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../UI/OpenAPI/python-flask-server/") 
        
           from openapi_server.models.response import Response

why server on arax.ncats.io /beta and /devED still works: start script does cd python-flask-server and runs python3 -m openapi_server, so python finds openapi_server from current dir. pytest runs from code/ARAX/test, from there it cant find it

RTX/code/UI/OpenAPI/python-flask-server/RTX_OpenAPI.start

Lines 18 to 23 in 744e574

    
           cd /mnt/data/orangeboard/$DEVAREA/RTX/code/UI/OpenAPI/python-flask-server 
        
           #export PATH=/mnt/data/python/Python-3.9.18/bin:$PATH # (old)for python 3.9 
        
           export PATH=/usr/bin:$PATH 
        
           exec python3 -u -m openapi_server 1>$LOGFILE 2>$ELOGFILE

#2904 add python-flask-server to sys.path in ARAX_response

fixed, one line in ARAX_response.py, same sys.path.append as other ARAX modules use:

RTX/code/ARAX/ARAXQuery/ARAX_response.py

Lines 7 to 8 in ef5679f

    
           sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../UI/OpenAPI/python-flask-server/") 
        
           from openapi_server.models.log_entry import LogEntry

now ARAX_response imports fine:

$ cd code/ARAX/ARAXQuery
$ python -c 'import ARAX_response' && echo "import ok"
import ok

and kp_info_cacher.py CLI works again:

$ cd Expand
$ python kp_info_cacher.py
The process with process ID 49968 has STARTED refreshing the KP info caches
  - Getting meta info from infores:retriever
The process with process ID 49968 has FINISHED refreshing the KP info caches

pytest now loads conftest and goes further, openapi_server error is gone:

$ cd ../../test
$ python -m pytest --collect-only -q -p no:cacheprovider 2>&1 | grep -c openapi_server
0

but it stops on next import error, same one for all 15 test modules (output trimmed):

$ python -m pytest --collect-only -q -p no:cacheprovider
...
_____________ ERROR collecting code/ARAX/test/test_ARAX_connect.py _____________
...
test_ARAX_connect.py:18: in <module>
    from ARAX_query import ARAXQuery
../ARAXQuery/ARAX_query.py:42: in <module>
    from response_cache import ResponseCache  #noqa: E402
../ResponseCache/response_cache.py:20: in <module>
    from flask import Flask,redirect
E   ModuleNotFoundError: No module named 'flask'
...
!!!!!!!!!!!!!!!!!!! Interrupted: 15 errors during collection !!!!!!!!!!!!!!!!!!!
71 tests collected, 15 errors in 3.52s

this flask one is next

why this fix: ARAX modules already add their paths with sys.path.append everywhere, this is how imports work in ARAX now

$ cd ../../..
$ grep -rn "sys.path.append" --include="*.py" code/ARAX | wc -l
     211
$ grep -rl "sys.path.append" --include="*.py" code/ARAX | wc -l
      90

and after fix no other module (outside tests) imports openapi_server without adding python-flask-server/ to path, this prints nothing:

$ grep -rlE "^\s*(from|import) openapi_server" --include="*.py" code/ARAX | grep -v /test/ | xargs grep -L "python-flask-server"

other options i looked at: pythonpath in pytest.ini fixes only pytest, CLI scripts like kp_info_cacher.py stay broken (CI also runs it in "Build KP info cache" step), and tests would hide the problem. proper packaging (pyproject, absolute imports) is real long term fix but its refactor of whole repo, probably not for this TRAPI 2.0 migration

after the openapi_server fix pytest stops on No module named 'flask'. and its not only pytest, the server itself dont start in fresh env (local app_server_config.json: port 5001, db check and background tasker off)

$ cd code/UI/OpenAPI/python-flask-server
$ python -m openapi_server
Loaded config file /Users/bazarkua/Work/RTX/code/UI/OpenAPI/python-flask-server/openapi_server/app_server_config.json
2026-10-14T03:50:34.121983: DEBUG: In BiolinkHelper init
2026-10-14T03:50:34.122572: DEBUG: Loading pickle file: /Users/bazarkua/Work/RTX/code/ARAX/BiolinkHelper/biolink_model_4.2.5.yaml
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/bazarkua/Work/RTX/code/UI/OpenAPI/python-flask-server/openapi_server/__main__.py", line 325, in <module>
    main()
  File "/Users/bazarkua/Work/RTX/code/UI/OpenAPI/python-flask-server/openapi_server/__main__.py", line 269, in main
    from openapi_server.apis.entity_api import router as EntityApiRouter
  File "/Users/bazarkua/Work/RTX/code/UI/OpenAPI/python-flask-server/openapi_server/apis/entity_api.py", line 36, in <module>
    importlib.import_module(name)
  File "/Users/bazarkua/.pyenv/versions/3.12.11/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bazarkua/Work/RTX/code/UI/OpenAPI/python-flask-server/openapi_server/impl/query_api_impl.py", line 59, in <module>
    import ARAX_query  # pylint: disable=import-outside-toplevel,import-error,wrong-import-position
    ^^^^^^^^^^^^^^^^^
  File "/Users/bazarkua/Work/RTX/code/ARAX/ARAXQuery/ARAX_query.py", line 42, in <module>
    from response_cache import ResponseCache  #noqa: E402
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bazarkua/Work/RTX/code/ARAX/ResponseCache/response_cache.py", line 20, in <module>
    from flask import Flask,redirect
ModuleNotFoundError: No module named 'flask'

flask is not installed bc Flask==2.2.5 was removed from requirements.txt in 5975cec with the move to FastAPI:

$ pip show flask
WARNING: Package(s) not found: flask

and this import is not used, Flask and redirect dont appear anywhere else in the file:

RTX/code/ARAX/ResponseCache/response_cache.py

Lines 18 to 22 in ef5679f

    
           import requests 
        
           import requests_cache 
        
           from flask import Flask,redirect 
        
           import copy 
        
           import multiprocessing

$ cd ../../../..
$ grep -n "Flask\|redirect" code/ARAX/ResponseCache/response_cache.py
20:from flask import Flask,redirect

why arax.ncats.io /beta and /devED still run: probably system python on the host still has Flask from master requirements (master still has Flask==2.2.5), i did not check the host. but any fresh install from trapi2.0 requirements (new venv, docker build, CI) would hit this

#2904 drop unused flask import from response_cache

fixed, removed unused from flask import Flask,redirect from response_cache.py:

RTX/code/ARAX/ResponseCache/response_cache.py

Lines 18 to 21 in d41de55

    
           import requests 
        
           import requests_cache 
        
           import copy 
        
           import multiprocessing

server now starts in fresh env:

$ cd code/UI/OpenAPI/python-flask-server
$ python -m openapi_server
Loaded config file /Users/bazarkua/Work/RTX/code/UI/OpenAPI/python-flask-server/openapi_server/app_server_config.json
2026-10-14T04:00:38.813012: DEBUG: In BiolinkHelper init
2026-10-14T04:00:38.813687: DEBUG: Loading pickle file: /Users/bazarkua/Work/RTX/code/ARAX/BiolinkHelper/biolink_model_4.2.5.yaml
Starting ARAX API on TCP port: 5001
INFO:     Started server process [52591]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:5001 (Press CTRL+C to quit)

and it answers (from other terminal):

$ curl -s http://localhost:5001/api/arax/v2.0/openapi.json | python3 -c 'import sys,json; i=json.load(sys.stdin)["info"]; print("title:", i.get("title")); print("version:", i.get("version")); print("x-trapi:", i.get("x-trapi",{}).get("version"))'
title: ARAX Translator Reasoner - TRAPI 2.0.0
version: 2.0.0
x-trapi: 2.0.0

pytest collects all tests now, no collection errors:

$ cd ../../../ARAX/test
$ python -m pytest --collect-only -q -p no:cacheprovider 2>&1 | tail -1
335 tests collected in 2.61s

why this fix and not adding Flask back to requirements.txt: ARAX dont use Flask anymore after the move to FastAPI, Flask and redirect were not used in this file, so adding Flask back only to keep a dead import alive makes no sense

only import flask left is in old connexion controllers/ (asyncquery_controller.py, query_controller.py), FastAPI server dont load them, thats why it starts without flask. can clean them up later

next I will try to run pytest locally on my MBP

first full run of pytest suite on trapi2.0 locally on my MBP, after the two import fixes above. all DBs from config_dbs.json are downloaded locally

$ cd code/ARAX/test
$ python -m pytest
...
collected 335 items

test_ARAX_connect.py Fs                                                  [  0%]
test_ARAX_expand.py ssssFsFssFssssssssssssssFFsFFFsFFsssssFFFFFFFsssssss [ 16%]
test_ARAX_filter_kg.py .sFFsssFsssFFFFFFF                                [ 27%]
test_ARAX_filter_results.py .FFsFsFFs                                    [ 30%]
test_ARAX_infer.py FFFFFFFFsFssssssss                                    [ 35%]
test_ARAX_json_queries.py FFsFs                                          [ 37%]
test_ARAX_messenger.py ..........F.F.                                    [ 41%]
test_ARAX_overlay.py sFFFsssssssssssssssssssss                           [ 48%]
test_ARAX_query.py F                                                     [ 48%]
test_ARAX_ranker.py sssssssssssss                                        [ 52%]
test_ARAX_resultify.py FFFFFFFs.sFFsFFFFFFsFF..ssFFFsssssFFFsssFssssss   [ 66%]
test_ARAX_standup_queries.py FFFssss                                     [ 68%]
test_ARAX_synonymizer.py .................................               [ 78%]
test_ARAX_translate.py FFFFFFFFssF                                       [ 82%]
test_ARAX_workflows.py FFssssssFsssssssFsssssss                          [ 89%]
test_ARAX_xcrg_connect.py ..F.FFFFF                                      [ 91%]
test_compute_ngd.py ........................                             [ 99%]
test_result_transformer.py FFF                                           [100%]
...
=========== 114 failed, 77 passed, 144 skipped, 1 warning in 41.95s ============

79 test cases fail in ARAXQuery.query() on the same line. it calls .to_dict() on query['message'], so looks like it expects pydantic Message object here, but tests pass message as plain dict (34) or dont have message key at all (45)

RTX/code/ARAX/ARAXQuery/ARAX_query.py

Lines 362 to 366 in d41de55

    
           eprint(f"*** {query}") 
        
           query_dict = query.copy() 
        
           query_dict['message'] = query['message'].to_dict() 
        
           eprint(json.dumps(query_dict, indent=2, sort_keys=True)) 
        
           result = self.examine_incoming_query(query_dict, mode=mode)

>       araxq.query(query)
test_ARAX_query.py:21: 
>           query_dict['message'] = query['message'].to_dict()
E           AttributeError: 'dict' object has no attribute 'to_dict'
../ARAXQuery/ARAX_query.py:364: AttributeError

>       [response, message] = _do_arax_query(query)
test_ARAX_workflows.py:84: 
test_ARAX_workflows.py:21: in _do_arax_query
>           query_dict['message'] = query['message'].to_dict()
E           KeyError: 'message'
../ARAXQuery/ARAX_query.py:364: KeyError

2 test cases fail in ARAX_messenger.add_qedge: it creates empty QEdge() first, but 2.0 model requires subject and object

>           messenger.add_qedge(response, parameters['parameters'])
test_ARAX_messenger.py:207: 
>       qedge = QEdge()
E       pydantic_core._pydantic_core.ValidationError: 2 validation errors for QEdge
E       subject
E         Field required [type=missing, input_value={}, input_type=dict]
E       object
E         Field required [type=missing, input_value={}, input_type=dict]
../ARAXQuery/ARAX_messenger.py:477: ValidationError

32 test cases fail on test data still in TRAPI 1.6 shape, 2.0 models validate it now:

Node without categories (9), or categories/ids given as string instead of list (7 Node, 2 QNode)
Edge without sources, knowledge_level, agent_type (9: 6 in the new Gandalf format update exposed a bug in blocklist filtering + xCRG ranking #2909 test from master, 3 in test_result_transformer.py)
mocked xCRG response with 1.6 node_bindings lists (3)
xCRG query graph routing (2), see below

>       kg_nodes = _create_nodes(kg_node_info)
test_ARAX_resultify.py:276: 
>           node = Node(categories=kg_node.get("categories"),
E           pydantic_core._pydantic_core.ValidationError: 1 validation error for Node
E           categories
E             Input should be a valid list [type=list_type, input_value='protein', input_type=str]
test_ARAX_resultify.py:51: ValidationError

xCRG routing: interpreter builds expand() plan instead of connect(action=xcrg)

>       assert response.data["araxi_commands"] == ["connect(action=xcrg)"]
E       AssertionError: assert ['expand()', ...results=500)'] == ['connect(action=xcrg)']
test_ARAX_xcrg_connect.py:170: AssertionError

looks like the reason is the test query graph. is_xcrg_mvp2_query() from xcrg package validates it with strict TRAPI 2.0 model (translator_tom), and test QG still has 1.6 qualifier_constraints plus auxiliary_graphs: {}, 2.0 dont allow it empty. when i convert both to 2.0 by hand it returns True. ARAX own envelope dont set auxiliary_graphs, so looks like test only

1 flaky: test_issue_2678 failed this run in query tracker (sqlalchemy.orm.exc.ObjectDeletedError in ARAX_query_tracker.check_ongoing_queries), in previous run same test failed in query() like the others. not checked

Support for Pathfinder TRAPI 2.0 #2904

I just confirmed that requirements.txt in the trapi2.0 branch installs cleanly in a fresh python3.12 virtualenv, with pip install -r requirements.txt. Thank you to whomever fixed it.

Great, thanks for confirming. I last fiddled with it after adding reasoner-validator 7.0.0 and seemed to get it into a good state.

Fix TRAPI 2.0 model errors breaking /query and creative mode #2904

Navigation Menu

Repository navigation

We need to upgrade all systems to TRAPI 2.0 in dev #2904

53 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	import datetime
	import os

	from openapi_server.models.log_entry import LogEntry

	sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../UI/OpenAPI/python-flask-server/")
	from openapi_server.models.response import Response

	cd /mnt/data/orangeboard/$DEVAREA/RTX/code/UI/OpenAPI/python-flask-server

	#export PATH=/mnt/data/python/Python-3.9.18/bin:$PATH # (old)for python 3.9
	export PATH=/usr/bin:$PATH

	exec python3 -u -m openapi_server 1>$LOGFILE 2>$ELOGFILE

	import requests
	import requests_cache
	from flask import Flask,redirect
	import copy
	import multiprocessing

	eprint(f"*** {query}")
	query_dict = query.copy()
	query_dict['message'] = query['message'].to_dict()
	eprint(json.dumps(query_dict, indent=2, sort_keys=True))
	result = self.examine_incoming_query(query_dict, mode=mode)

We need to upgrade all systems to TRAPI 2.0 in dev #2904

Description

Activity

saramsey commented on Oct 7, 2026

saramsey commented on Oct 7, 2026

saramsey commented on Oct 7, 2026

53 remaining items

bazarkua commented on Oct 14, 2026

bazarkua commented on Oct 14, 2026

bazarkua commented on Oct 14, 2026

bazarkua commented on Oct 14, 2026

bazarkua commented on Oct 14, 2026

saramsey commented on Oct 21, 2026

edeutsch commented on Oct 21, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions