-
Notifications
You must be signed in to change notification settings - Fork 5k
bundler: chain inline input sourcemaps through to output #30539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
robobun
wants to merge
29
commits into
main
Choose a base branch
from
farm/eb1afa62/chain-input-sourcemaps
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
31e2ddc
bundler: chain inline input sourcemaps through to output
robobun ee030e8
[autofix.ci] apply automated fixes
autofix-ci[bot] 87f50bc
test: cover plugin onLoad with inline sourcemap (#6173)
robobun e84d5c4
address review findings: OOM, leak, test guards
robobun ac7b108
sourcemap: anchor findSourceMappingURL to the final line
robobun 9eca1d4
address 3 review findings from claude[bot]
robobun 47757dc
ci: retrigger
robobun a10f2b2
test: gate chained-sourcemap tests behind describe.todo
robobun 02b8ec8
bundler: chain inline input sourcemaps through to output (Rust port)
robobun 35cbe17
[autofix.ci] apply automated fixes
autofix-ci[bot] 4983e19
sourcemap: trim whitespace on both sides of the URL after sourceMappi…
robobun 6dd5c39
bundler: init Graph::InputFile.input_source_map in Default impl
robobun 2b61ec7
sourcemap: drop unused Mapping import from InputSourceMap.rs
robobun f5187ac
bundler: hoist source_map option before get_ast to avoid stacked-borr…
robobun 1ae23e1
sourcemap: drop redundant mut on estr bindings (-D unused-mut)
robobun 8266423
sourcemap: hint find_line_with_hint from intermediate line, not remapped
robobun 60af998
[autofix.ci] apply automated fixes
autofix-ci[bot] 5b10b6e
sourcemap: align comments with main's port-note cleanup
robobun ee552a7
sourcemap: route write_sources_for paths through source_map_relative_…
robobun cfe8ba5
sourcemap: use Number::value() accessor (field now private)
robobun b83f1e7
bundler: cap inline-sourcemap source names; isolate chain tests
robobun 37dad95
test: run inline-sourcemap chain suite concurrently
robobun 3a6030d
test: size oversized-name case past the largest platform MAX_PATH_BYTES
robobun c9447b7
sourcemap: adapt InputSourceMap to renamed json parse and Result-base…
robobun 5f22626
sourcemap: read inline maps through the tape-based JSON accessors
robobun 7d009dd
sourcemap: trim explanatory comments to their load-bearing core
robobun 27ec2ea
sourcemap: use bun_core::strings helpers for byte search (source lint)
robobun 39e1360
bundler: skip inline-map scan under DevServer; pass URL-schemed sourc…
robobun 2d79d79
bundler: emit inner source names verbatim for virtual-namespace modules
robobun File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1044,89 +1044,90 @@ impl<'a> LinkerContext<'a> { | |
|
|
||
| let sources = self.parse_graph().input_files.items_source(); | ||
| let quoted_source_map_contents = self.graph.files.items_quoted_source_contents(); | ||
| // DevServer's stitcher (`SourceMapStore::join_vlq`) assumes one | ||
| // `sources[]` slot per input, so chaining is gated to `Bun.build`. | ||
|
robobun marked this conversation as resolved.
|
||
| let input_source_maps: Option<&[Option<Box<bun_sourcemap::InputSourceMap>>]> = | ||
| if self.dev_server.is_none() { | ||
| Some(self.parse_graph().input_files.items_input_source_map()) | ||
| } else { | ||
| None | ||
| }; | ||
|
|
||
| // Entries in `results` do not 1:1 map to source files, the mapping | ||
| // is actually many to one, where a source file can have multiple chunks | ||
| // in the sourcemap. | ||
| // | ||
| // This hashmap is going to map: | ||
| // `source_index` (per compilation) in a chunk | ||
| // --> | ||
| // Which source index in the generated sourcemap, referred to | ||
| // as the "mapping source index" within this function to be distinct. | ||
| // Many-to-one: a source file can own several chunks. Maps each | ||
| // compilation `source_index` to its base index in the generated | ||
| // `sources[]`; a file with an inline map spans | ||
| // `base ..= base + external_source_names.len`. | ||
|
robobun marked this conversation as resolved.
|
||
| let mut source_id_map: ArrayHashMap<u32, i32> = ArrayHashMap::new(); | ||
|
|
||
| let source_indices = results.items_source_index(); | ||
|
|
||
| j.push_static(b"{\n \"version\": 3,\n \"sources\": ["); | ||
| let mut next_mapping_source_index: i32 = 0; | ||
| if !source_indices.is_empty() { | ||
| { | ||
| let index = source_indices[0]; | ||
| let path = &sources[index as usize].path; | ||
| source_id_map.put_no_clobber(index, 0)?; | ||
|
|
||
| // Note: the relative path lives in a local owned buffer | ||
| // (drops at scope exit). | ||
| let rel_path_storage; | ||
| let pretty: &[u8] = if path.is_file() { | ||
| rel_path_storage = Self::source_map_relative_path(chunk_abs_dir, path.text)?; | ||
| &rel_path_storage | ||
| } else { | ||
| path.pretty | ||
| }; | ||
|
|
||
| let mut quote_buf = MutableString::init(pretty.len() + 2)?; | ||
| js_printer::quote_for_json(pretty, &mut quote_buf, false)?; | ||
| // `to_default_owned` moves the buffer into the joiner | ||
| // (joiner owns it until `done`). | ||
| j.push_owned(quote_buf.to_default_owned()); | ||
| } | ||
|
|
||
| let mut next_mapping_source_index: i32 = 1; | ||
| for &index in &source_indices[1..] { | ||
| for (chunk_i, &index) in source_indices.iter().enumerate() { | ||
| let gop = source_id_map.get_or_put(index)?; | ||
| if gop.found_existing { | ||
| continue; | ||
| } | ||
|
|
||
| *gop.value_ptr = next_mapping_source_index; | ||
| next_mapping_source_index += 1; | ||
|
|
||
| let path = &sources[index as usize].path; | ||
|
|
||
| let rel_path_storage; | ||
| let pretty: &[u8] = if path.is_file() { | ||
| rel_path_storage = Self::source_map_relative_path(chunk_abs_dir, path.text)?; | ||
| &rel_path_storage | ||
| } else { | ||
| path.pretty | ||
| // `1` for the intermediate input, plus one slot per inner | ||
| // source listed in its `sourceMappingURL`. | ||
|
robobun marked this conversation as resolved.
|
||
| let inner: Option<&bun_sourcemap::InputSourceMap> = | ||
| input_source_maps.and_then(|m| m[index as usize].as_deref()); | ||
| let expansion: i32 = 1 + match inner { | ||
| Some(ism) => { | ||
| i32::try_from(ism.map.external_source_names.len()).expect("int cast") | ||
| } | ||
| None => 0, | ||
| }; | ||
|
|
||
| let mut quote_buf = MutableString::init(pretty.len() + ", ".len() + 2)?; | ||
| quote_buf.append_assume_capacity(b", "); | ||
| js_printer::quote_for_json(pretty, &mut quote_buf, false)?; | ||
| j.push_owned(quote_buf.to_default_owned()); | ||
| next_mapping_source_index += expansion; | ||
|
|
||
| write_sources_for( | ||
| &mut j, | ||
| chunk_abs_dir, | ||
| &sources[index as usize].path, | ||
| inner, | ||
| chunk_i > 0, | ||
| )?; | ||
| } | ||
| } | ||
|
|
||
| j.push_static(b"],\n \"sourcesContent\": ["); | ||
|
|
||
| let source_indices_for_contents = source_id_map.keys(); | ||
| if !source_indices_for_contents.is_empty() { | ||
| j.push_static(b"\n "); | ||
| j.push_static( | ||
| quoted_source_map_contents[source_indices_for_contents[0] as usize] | ||
| .as_deref() | ||
| .unwrap_or(b""), | ||
| ); | ||
|
|
||
| for &index in &source_indices_for_contents[1..] { | ||
| j.push_static(b",\n "); | ||
| j.push_static( | ||
| quoted_source_map_contents[index as usize] | ||
| let mut emitted_contents: usize = 0; | ||
| for &index in source_indices_for_contents.iter() { | ||
| // Slot 0: the intermediate input file's contents (already | ||
| // JSON-quoted by `compute_quoted_source_contents`). | ||
|
robobun marked this conversation as resolved.
|
||
| { | ||
| let sep: &[u8] = if emitted_contents == 0 { | ||
| b"\n " | ||
| } else { | ||
| b",\n " | ||
| }; | ||
| j.push_static(sep); | ||
| let content = quoted_source_map_contents[index as usize] | ||
| .as_deref() | ||
| .unwrap_or(b""), | ||
| ); | ||
| .unwrap_or(b"null"); | ||
| j.push_static(if content.is_empty() { b"null" } else { content }); | ||
| emitted_contents += 1; | ||
| } | ||
| // Slots 1..N: inner sources' contents, if any. | ||
| if let Some(ism) = input_source_maps.and_then(|m| m[index as usize].as_deref()) { | ||
| for content in ism.sources_content.iter() { | ||
| j.push_static(b",\n "); | ||
| if !content.is_empty() { | ||
| let mut quote_buf = MutableString::init(content.len() + 2)?; | ||
| js_printer::quote_for_json(content, &mut quote_buf, false)?; | ||
| j.push_owned(quote_buf.to_default_owned()); | ||
| } else { | ||
| j.push_static(b"null"); | ||
| } | ||
| emitted_contents += 1; | ||
| } | ||
| } | ||
| } | ||
| } | ||
| j.push_static(b"\n ],\n \"mappings\": \""); | ||
|
|
@@ -1166,7 +1167,9 @@ impl<'a> LinkerContext<'a> { | |
| )?; | ||
|
|
||
| prev_end_state = chunk.end_state; | ||
| prev_end_state.source_index = mapping_source_index; | ||
| // `chunk.end_state.source_index` is chunk-relative (0 without an | ||
| // inline map); rebase it onto this file's slot base. | ||
|
robobun marked this conversation as resolved.
|
||
| prev_end_state.source_index = mapping_source_index + chunk.end_state.source_index; | ||
| prev_column_offset = chunk.final_generated_column; | ||
|
|
||
| if prev_end_state.generated_line == 0 { | ||
|
|
@@ -1210,6 +1213,85 @@ impl<'a> LinkerContext<'a> { | |
| } | ||
| } | ||
|
|
||
| /// Emit one outer source's quoted path plus its chained inner paths, in | ||
| /// the slot layout `Chunk::Builder` emits against: slot 0 = the outer | ||
| /// file, slots 1..N = inner `sources[i]`. | ||
|
robobun marked this conversation as resolved.
|
||
| fn write_sources_for( | ||
| joiner: &mut StringJoiner, | ||
| chunk_abs_dir: &[u8], | ||
| outer_path: &bun_paths::fs::Path, | ||
| input_map: Option<&bun_sourcemap::InputSourceMap>, | ||
| leading_comma: bool, | ||
| ) -> Result<(), BunError> { | ||
| // 1) the intermediate input. | ||
| let rel_path_storage; | ||
| let pretty: &[u8] = if outer_path.is_file() { | ||
| rel_path_storage = LinkerContext::source_map_relative_path(chunk_abs_dir, outer_path.text)?; | ||
| &rel_path_storage | ||
| } else { | ||
| outer_path.pretty | ||
| }; | ||
| { | ||
| let mut quote_buf = MutableString::init(pretty.len() + ", ".len() + 2)?; | ||
| if leading_comma { | ||
| quote_buf.append_assume_capacity(b", "); | ||
| } | ||
| js_printer::quote_for_json(pretty, &mut quote_buf, false)?; | ||
| joiner.push_owned(quote_buf.to_default_owned()); | ||
| } | ||
|
|
||
| // 2) inner sources: resolve each against the intermediate's dir, then | ||
| // re-relativize to `chunk_abs_dir` for the emitted JSON. | ||
|
robobun marked this conversation as resolved.
|
||
| if let Some(ism) = input_map { | ||
| let emit = |joiner: &mut StringJoiner, p: &[u8]| -> Result<(), BunError> { | ||
| let mut quote_buf = MutableString::init(p.len() + ", ".len() + 2)?; | ||
| quote_buf.append_assume_capacity(b", "); | ||
| js_printer::quote_for_json(p, &mut quote_buf, false)?; | ||
| joiner.push_owned(quote_buf.to_default_owned()); | ||
| Ok(()) | ||
| }; | ||
| // A non-file intermediate (plugin virtual module) has no directory | ||
| // to resolve against; emit inner names verbatim. | ||
|
Comment on lines
+1253
to
+1254
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code |
||
| if !outer_path.is_file() { | ||
| for name in ism.map.external_source_names.iter() { | ||
| emit(joiner, name.as_ref())?; | ||
| } | ||
| return Ok(()); | ||
| } | ||
| let base_dir = bun_paths::resolve_path::dirname::<bun_paths::resolve_path::platform::Auto>( | ||
| outer_path.text, | ||
| ); | ||
| let mut join_buf = bun_paths::path_buffer_pool::get(); | ||
| for name in ism.map.external_source_names.iter() { | ||
| let name: &[u8] = name.as_ref(); | ||
| // The spec allows URLs in `sources[]` (e.g. `webpack:///src/a.ts`); | ||
| // path-joining would destroy the scheme, so pass them through. | ||
|
robobun marked this conversation as resolved.
|
||
| if bun_core::strings::index_of(name, b"://").is_some() { | ||
| emit(joiner, name)?; | ||
| continue; | ||
| } | ||
| if bun_paths::resolve_path::Platform::AUTO.is_absolute(name) { | ||
| let rel = LinkerContext::source_map_relative_path(chunk_abs_dir, name)?; | ||
| emit(joiner, &rel)?; | ||
| continue; | ||
| } | ||
| // The checked join returns `None` on overflow (adversarial | ||
| // map); emit the raw, spec-valid name instead of panicking. | ||
|
robobun marked this conversation as resolved.
|
||
| match bun_paths::resolve_path::join_abs_string_buf_checked::< | ||
| bun_paths::resolve_path::platform::Auto, | ||
| >(base_dir, join_buf.as_mut_slice(), &[name]) | ||
| { | ||
| Some(abs_path) => { | ||
| let rel = LinkerContext::source_map_relative_path(chunk_abs_dir, abs_path)?; | ||
| emit(joiner, &rel)?; | ||
| } | ||
| None => emit(joiner, name)?, | ||
| } | ||
| } | ||
|
claude[bot] marked this conversation as resolved.
|
||
| } | ||
| Ok(()) | ||
| } | ||
|
|
||
| #[derive(Clone, Copy, PartialEq, Eq)] | ||
| pub(crate) enum ScanCssImportsResult { | ||
| Ok, | ||
|
|
@@ -2193,6 +2275,14 @@ impl<'a> LinkerContext<'a> { | |
| // SAFETY: `self.mangled_props` is not mutated during printing; detached borrow | ||
| // outlives only this call (see above). | ||
| unsafe { bun_ptr::detach_lifetime_ref(&self.mangled_props) }; | ||
| // DevServer's stitcher assumes one `sources[]` slot per file; | ||
| // chaining is gated to the `Bun.build` path. | ||
|
robobun marked this conversation as resolved.
|
||
| let input_source_map: Option<&bun_sourcemap::InputSourceMap> = if self.dev_server.is_none() | ||
| { | ||
| parse_graph.input_files.items_input_source_map()[source_index.get() as usize].as_deref() | ||
| } else { | ||
| None | ||
| }; | ||
|
|
||
| let print_options = js_printer::Options { | ||
| bundling: true, | ||
|
|
@@ -2241,6 +2331,7 @@ impl<'a> LinkerContext<'a> { | |
| } else { | ||
| None | ||
| }, | ||
| input_source_map, | ||
| mangled_props: Some(mangled_props), | ||
| module_info, | ||
| ..Default::default() | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.