Recursion

[firefart]

No description provided.

[Copilot]

🟡 Changes recommended

The recursion orchestration currently fails to consider recursive targets found by results from some success-driven follow-up guesses, which can cause missed recursion targets in real scans.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds recursive scanning support to Gobuster’s core runner and dir mode, enabling directory discoveries to seed follow-up scans with configurable depth and target limits.

Changes:

• Introduces recursion-related options and interfaces (RecursivePlugin, RecursiveResult) in libgobuster, plus orchestration logic in the main run loop.
• Extends gobusterdir to emit recursive targets for directory candidates and to support retargeting between scans.
• Updates the dir CLI command with --recursive, --recursion-depth, and --recursion-max-targets, and refreshes README usage/docs.

File summaries

File	Description
README.md	Reworked documentation and added --recursive mention for dir mode usage.
libgobuster/options.go	Adds recursion configuration fields to global options.
libgobuster/libgobuster.go	Implements recursive run orchestration and adapts success handling to carry results.
libgobuster/libgobuster_test.go	Adds tests covering recursion depth/limit behavior and plugin support gating.
libgobuster/interfaces.go	Defines recursion-capable plugin/result interfaces.
gobusterdir/result.go	Extends dir results to optionally expose a recursion target.
gobusterdir/gobusterdir.go	Adds target switching (SetTarget) and emits recursion targets for directory candidates.
cli/dir/dir.go	Adds recursion flags and validates recursion depth/target limits.

Review details

Suppressed comments (1)

cli/dir/dir.go:62

• Error message grammar: use “greater than or equal to 0” instead of “bigger or equal to 0”.

	if globalOpts.RecursionMaxTargets < 0 {
		return errors.New("recursion-max-targets must be bigger or equal to 0")
	}

• Files reviewed: 8/8 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The current directory-candidate heuristic can incorrectly recurse into file hits by default (no extensions configured), leading to wrong targets and unnecessary scan amplification.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

gobusterdir/gobusterdir.go:412

• isDirectoryCandidate returns true when no extensions were configured (the default), so any successful hit (including obvious files like robots.txt) will be treated as a directory and queued for recursion (as .../robots.txt/). This can create incorrect recursion targets and unnecessary scan amplification.

	return true

• Files reviewed: 8/8 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Recursive scans can lose query parameters, overwrite saved bodies, and incorrectly recurse into files.

Review details

Suppressed comments (5)

Previously missed (4) — in code that hasn't changed since the last review.

gobusterdir/gobusterdir.go:122

• Clearing RawQuery drops base query parameters from every recursive level. Directory mode otherwise preserves the initial query on each request, so a target such as ...?token=... works at the root but loses the token after SetTarget; preserve the query and clear only the fragment.

This issue also appears on line 388 of the same file.

	u.RawQuery = ""
	u.Fragment = ""

README.md:82

• The blacklist does not override the positive list: cli/dir/dir.go:103-105 rejects the command when both are non-empty. Please describe the options as mutually exclusive so users understand why -b "" is required.

The status-code blacklist defaults to `404` and overrides the positive list, so
explicitly clear it with `-b ""` when using `-s`.

gobusterdir/gobusterdir.go:370

• Recursive scans reuse the same wordlist, but response bodies are still named only from entity and status at lines 338-341. The same word under two targets therefore silently overwrites the earlier body (for example, /admin/index and /api/index both produce index_200.html); include the recursive base path or full-URL hash in the filename.

			if d.globalopts.Recursion {
				displayPath = fmt.Sprintf("%s%s", d.options.URL.Path, entity)
			}

gobusterdir/gobusterdir.go:411

• This identifies files only when their suffix was also supplied via -x. A wordlist entry such as robots.txt without -x, or a successful .bak generated by --discover-backup, is treated as a directory and recursively scanned as robots.txt/ or .bak/. Track whether a guess is a directory candidate when creating it rather than inferring that from the configured extension set.

func (d *GobusterDir) isDirectoryCandidate(word string) bool {
	for ext := range d.options.ExtensionsParsed.Set {
		if strings.HasSuffix(word, "."+ext) {
			return false
		}
	}
	return true

gobusterdir/gobusterdir.go:389

• This independently removes the query while constructing the next target, so fixed query parameters used by the initial scan disappear for child scans. Preserve RawQuery here; SetTarget should do the same.

				recursionURL.RawQuery = ""
				recursionURL.Fragment = ""

• Files reviewed: 8/8 changed files
• Comments generated: 0 new
• Review effort level: Balanced