Fix PaddleOCR 2.9+ args, Box Sorting, and pin Transformers (Florence-2 fix)

[Shivaansh Gusain (ShivaanshGusain)]

Summary of Changes

This PR addresses two compatibility issues encountered when setting up the environment with recent library versions (specifically PaddleOCR v2.9.1).

1. Fix PaddleOCR Initialization
   Newer versions of paddleocr have deprecated/removed arguments like max_batch_size, use_gpu, and use_dilation from the constructor. Keeping them causes a ValueError crash on startup.
   Change: Updated PaddleOCR() initialization to rely on default argument parsing, which correctly auto-detects GPU support.

2. Box Sorting
   Occasionally, filtered_boxes contains mixed types (dictionaries and raw lists) depending on the detection results. This causes the sorted() function (line ~435) to crash with an AttributeError because raw lists do not have keys.
   Change: Added a safety check loop (safe_boxes) to ensure all elements in filtered_boxes are standardized dictionaries before sorting.

3. Fix Florence-2 Inference Crash
   The current dependency resolution installs a version of transformers (v4.41+) that is incompatible with the custom modeling_florence2.py, causing an AttributeError: 'NoneType' object has no attribute 'shape'. Change: Pinned transformers==4.40.0 in requirements.txt to ensure stability.

Testing

• Verified environment setup on Windows with Python 3.10 and PaddleOCR 2.9.1.
• Validated that the model loads and performs inference without crashing.

[Shivaansh Gusain (ShivaanshGusain)]

@microsoft-github-policy-service agree

[Shivaansh Gusain (ShivaanshGusain)]

Hi Alexey Taymanov (@ataymano), this is the fix for setting up the environment with the latest version of paddleocr (v2.9.1) -

Fixed PaddleOCR Initialization: Newer versions of paddleocr have deprecated or removed the max_batch_size, use_gpu, and use_dilation arguments. Passing them was causing a ValueError on startup. I’ve updated the initialization to rely on the default argument parsing, which correctly handles GPU detection automatically.

I noticed that filtered_boxes occasionally contained mixed types (dictionaries and raw lists) depending on the detection results, which caused the sorted() function to crash with an AttributeError. I added a safety check to standardize all elements to dictionaries before sorting.

I verified these changes on Windows with Python 3.11 and PaddleOCR 2.9.1. The model now loads correctly and performs inference without crashing.

[Jorge (jomach)]

See #354

[Shivaansh Gusain (ShivaanshGusain)]

See #354

The crash in your logs is actually coming from the Florence-2 model, not Paddle. The AttributeError: 'NoneType' object... error occurs because recent versions of the Transformers library break the custom model code.

I have updated requirements.txt in the PR above to pin the correct version.

Quick fix -
Run' pip install transformers==4.40.0' in your environment, and it should work immediately.

[Jorge (jomach)]

I also had to add on the utils.py attn_implementation="eager" on :

if device == 'cpu':
            model = AutoModelForCausalLM.from_pretrained(model_name_or_path, torch_dtype=torch.float32, trust_remote_code=True, attn_implementation=attn_implementation)
        else:
            model = AutoModelForCausalLM.from_pretrained(model_name_or_path, torch_dtype=torch.float16, trust_remote_code=True, attn_implementation=attn_implementation).to(device)

[XingdongZeng]

Shivaansh Gusain (@ShivaanshGusain) i encounter this issue with your repo, could you help to fix this

&expand=false "HTTP/1.1 200 OK" Traceback (most recent call last): File "D:\02WORKTEMP\02CODEREPO\06PYTHON\OmniParser\gradio_demo.py", line 16, in <module> caption_model_processor = get_caption_model_processor(model_name="florence2", model_name_or_path="weights/icon_caption_florence") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\02WORKTEMP\02CODEREPO\06PYTHON\OmniParser\util\utils.py", line 57, in get_caption_model_processor processor = AutoProcessor.from_pretrained("microsoft/Florence-2-base", trust_remote_code=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python\Lib\site-packages\transformers\models\auto\processing_auto.py", line 407, in from_pretrained return processor_class.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python\Lib\site-packages\transformers\processing_utils.py", line 1404, in from_pretrained return cls.from_args_and_dict(args, processor_dict, **instantiation_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python\Lib\site-packages\transformers\processing_utils.py", line 1171, in from_args_and_dict processor = cls(*args, **valid_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\z00443194\.cache\huggingface\modules\transformers_modules\microsoft\Florence_hyphen_2_hyphen_base\5ca5edf5bd017b9919c05d08aebef5e4c7ac3bac\processing_florence2.py", line 89, in __init__ tokenizer.additional_special_tokens + \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python\Lib\site-packages\transformers\tokenization_utils_base.py", line 1293, in __getattr__ raise AttributeError(f"{self.__class__.__name__} has no attribute {key}") AttributeError: TokenizersBackend has no attribute additional_special_tokens. Did you mean: 'add_special_tokens'?

[Shivaansh Gusain (ShivaanshGusain)]

XingdongZeng
Thank you for reporting this.

This crash happens because the custom Florence-2 model code is incompatible with the latest versions of the transformers library. I recently updated the requirements.txt to pin the stable version

You can either run the requirements files to install the correct module versions.

Or downgrade transformers module to 4.40.0

'pip install transformers==4.40.0'

[Jorge (jomach)]

Can we merge this ?

[Shivaansh Gusain (ShivaanshGusain)]

Jorge (@jomach) I would love to get this merged into the official repository.
I can't merge this myself. Do you have a way to get this merged in the official repository?

[Jorge (jomach)]

Shivaansh Gusain (@ShivaanshGusain) I don't :( ping the repo owners

[Copilot]

Pull request overview

This PR aims to improve runtime compatibility with newer dependency versions by updating PaddleOCR initialization, making OCR/Yolo box post-processing more robust, and pinning transformers to a known-good version for Florence-2.

Changes:

• Removed deprecated PaddleOCR constructor arguments to avoid startup failures on PaddleOCR 2.9+.
• Added normalization around filtered_boxes to prevent sorting crashes when mixed element types appear.
• Pinned transformers==4.40.0 (and clarified OpenCV/Paddle installation notes) to avoid Florence-2 inference crashes.

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated 2 comments.

File	Description
util/utils.py	Updates PaddleOCR init, adjusts BoxAnnotator import, hardens box sorting logic, and modifies get_som_labeled_img output handling.
requirements.txt	Pins transformers==4.40.0 and adds guidance/comments around OpenCV and Paddle CPU vs GPU installs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Shivaansh Gusain (ShivaanshGusain)]

Hi ahmed-awadallah / Adam (@adamlu123), just checking in on this PR!
I've fixed the Florence-2 crashes and the PaddleOCR 2.9+ initialization bugs. The automated Copilot review has also been resolved. Let me know if you need me to make any adjustments before this can be merged!