Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 147 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,153 @@
# Changelog

All notable changes to this project will be documented in this file.
## [0.9.3] - 2026-09-10

### Features

- add opt-in dynamic attack history
- webui fetch and convert script
- webui edit seeds and datasets
- enhance session management for webui executions and persist state
- implement manual job handling and UI for dataset testing
- add --version arg
- add manual prompting functionality
- webui plugin workshop
- add SQLite database support for job persistence
- add rerun functionality for jobs
- implement plugin/attack refresh and truncation settings page
- add lazy loading for plugin and attack moduletags
- spikee ui - phase 4 (test)
- spikee ui - phase 3 (generate + jobs)
- spikee ui - phase 1 (results)
- spikee ui - phase 0 (wireframe)
- implement response validation and enhance debug logging in providers
- module debug command
- add homoglyph encoding plugin (Unicode confusables)
- add spikee init error for sourcecode
- add interactive docs
- openai sts
- add content wrapper to tester
- add content wrapper to generator and plugins. Updated module description and option type hints.
- digraphic lang and updated module tags (#91)
- aws transcribe stt + linting + aws profiles to polly
- add provider timeouts (#95)
- bedrock sso provider
- implement streaming for TTS providers
- add aws polly tts provider
- add TTS plugin
- add elevenlabs tts and stt providers
- add openai tts and stt providers
- add text2image plugin
- Suppress warnings during model loading in OpusTranslator
- Update attack module tags
- Improve plugin module tag taxonomies and list representations
- add digraphic language plugin
- local translation plugin (#85)
- provider modules (agent-framework) and refactoring type hints (#83)
- refactor web-viewer, add refresh options (#82)
- goat attack (#77)
- refactor LLM targets and add billing
- Streamline TogetherAI content retrieval, fix DeepSeek model parsing, update AWS Bedrock test model, and adjust Azure LLM deployment naming.
- improve prefix and suffix handling in dataset generation (#74)
- add LLMWrapper
- Add tests for invalid model names across inference, attack, and judge targets, ensuring proper error handling, and refactor judge option passing in tests.
- Add OpenRouter API target, integrate Azure, Groq, and Deepseek with litellm, and update inference tests.
- Add guardrails to progress bar, remove guardrail print statements (#73)
- plugin piping, new plugins (google_translate, shortener, mask) and improvements to splat (#72)
- `spikee list` improvements, LLM-Driven Plugins and Echo Chamber Attack (#69)
- web-viewer and StandardisedConversations (#64)
- add LLM-based attacks (#65)
- use environment vars for ollama timeout and retries (#58)
- multi-option judge args, and llm utilities
- Modified extract to allow multiple --custom-search flags and negative searches
- add dynamic CLI option support for judges, targets, plugins, and attacks
- Version 0.2 - Enhanced Testing with Attacks & Judges

### Fixes

- add SKIP_PYPI environment variable to control PyPI publication in release workflow
- refine conditions for lint and functional test jobs in workflow
- pin ruff version to 0.16.6 in release and test workflows
- declare tokenizer dependency for random suffix attack
- debug plugin option key and error handling in OpusTranslator (#122)
- add future annotations import to crescendo.py to fix test bug
- force progress bar refresh after early completion updates in attack modules and tester
- ensure standalone entries preserve instruction_type and jailbreak_type from seed data
- delay progress bar updates in echo_chamber attack to prevent premature incrementing
- webui bugs
- btn ui
- Ensure asyncio event loops are properly closed in worker threads
- update result viewer function references from string_to_colour to text_to_colour
- improve async call cleanup by forcing garbage collection and handling pending tasks
- typing of static multi-turn dataset generation
- judge list flattening bug
- tester original attack input error
- list bug
- process_conversation bug
- handle plugin transformation errors and improve AWS credential handling
- async any-llm bug
- content wrapper generator bug
- provider hinting
- generation content, update tests
- any-llm asyncio conflict with tester multiprocessing (#93)
- default llm judge model (#92)
- update default model IDs
- linting
- linting
- generation progress bar and plugin only
- Re-raise `NotFoundError` instead of exiting and update model names in inference tests.
- objective judge bug
- offline judge bug
- correctly handle guardrail target output (boolean) and add a corresponding functional test. (#71)
- imports that break release (#67)
- bug in custom search - enforce string (#61)
- shared dict - single-turn check bug (#59)
- use correct ChatOllama parameters (#56)
- ollama model selection for ollama targets (#55)
- display options for judges in `spikee list judges`
- Correctly handle resume file selection for multiple datasets and progress bar totals.
- progress bar calculation to use items to process instead of full dataset. (#51)
- remove leftover debug prints
- custom extract none bug
- remove debug prints
- custom extract none bug
- docstring warning
- add missing toml python dependency
- check for older custom judge signature

### Changes

- refactor plugin and attack handling in generation UI
- update 'spikee viewer' to 'spikee webui'
- add `_invoke` function to providers to enable built-in debugging and management.
- add MessageHint
- Update module description in homoglyph.py
- add GTR to breakdowns and update overview labels
- minimised content wrappers
- fix type hinting and add content type checks for providers
- move profiles to any-llm bedrock provider
- update naming scheme and add docs
- Enhance error handling in spikee list and add multi-modal tags
- add Modules to viewers (#84)
- add Modules to viewers
- add model not found error
- remove extract dataset
- remove extract dataset
- update message formats + llm bugs
- update llm_judge_objective prompt
- increase echo chamber efficiency
- remove typo in cybersec dataset (#63)
- add 'offline' to example LLM models list (#62)
- modify llm judge options to allow for more models and providers
- add quiet switch (#38)

### Datasets

- add cybersec-2026-01, make cybersec-2025-04 legacy (#66)
- add toxic-chat
- fixed typo in sysmsg-extraction dataset, unneeded exclude_from_transformation_regex

## [0.9.2] - 2026-09-10

### Features
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
</p>
</div>

_Version: 0.9.2_
_Version: 0.9.3_


Developed by Reversec Labs, `spikee` is a toolkit for assessing the resilience of LLMs, guardrails, and applications against prompt injection and jailbreaking. Spikee's strength is its modular design, which allows for easy customization of every part of the testing process.
Expand Down
8 changes: 4 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "spikee"
version = "0.9.2"
version = "0.9.3"
description = "Spikee - Simple Prompt Injection Kit for Evaluation and Exploitation"
readme = "README.md"
requires-python = ">=3.11"
Expand Down Expand Up @@ -33,12 +33,12 @@ spikee = "spikee.cli:main"
[tool.ruff]
include = [ "**/*.py",]

[tool.setuptools.package-data]
spikee = [ "data/**/*", "viewer/**/*",]

[tool.ruff.lint]
ignore = [ "S110",]

[tool.setuptools.package-data]
spikee = [ "data/**/*", "viewer/**/*",]

[tool.setuptools.packages.find]
include = [ "spikee*",]
exclude = [ "datasets*", "results*", "targets*", "plugins*", "attacks*", "judges*", "templates*", "utilities*",]
2 changes: 1 addition & 1 deletion spikee/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.9.2"
__version__ = "0.9.3"
Loading