Skip to content

Switch to Cell::update, slice::{split_first_chunk,split_off}, where appropriate#6974

Merged
youknowone merged 2 commits intoRustPython:mainfrom
coolreader18:newish-apis
Feb 3, 2026
Merged

Switch to Cell::update, slice::{split_first_chunk,split_off}, where appropriate#6974
youknowone merged 2 commits intoRustPython:mainfrom
coolreader18:newish-apis

Conversation

@coolreader18
Copy link
Member

@coolreader18 coolreader18 commented Feb 3, 2026

Summary by CodeRabbit

  • New Features

    • Added fixed-size array reading capability to the marshaling API for improved data retrieval
  • Bug Fixes

    • Enhanced UTF-8 surrogate decoding with improved error handling for incomplete byte sequences
    • Strengthened robustness of C type buffer reading and data extraction across operations
    • Refined recursion depth management for enhanced reliability during nested operations

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 3, 2026

📝 Walkthrough

Walkthrough

This PR refactors multiple crates to use safer, more idiomatic Rust patterns for buffer/slice operations. Changes replace manual bounds checks and unwraps with first_chunk(), map_or(), and other standard library methods, along with adding a new read_array API to the marshalling trait and improving scope-guarded recursion depth management.

Changes

Cohort / File(s) Summary
Cell operations
crates/common/src/lock/cell_lock.rs
Replaced get() and set() calls with a single Cell::update() closure for decrementing reader count in unlock_shared.
Buffer/slice handling refactoring
crates/vm/src/stdlib/ctypes/array.rs, crates/vm/src/stdlib/ctypes/base.rs, crates/vm/src/stdlib/ctypes/function.rs, crates/vm/src/stdlib/ctypes/simple.rs
Consolidated manual length checks and slice conversions into safer first_chunk() and map_or() patterns for fixed-size byte extraction across integer and floating-point type reads.
Marshalling API enhancement
crates/compiler-core/src/marshal.rs
Added new public read_array<const N: usize>() method to Read trait and implemented it for &[u8] to read fixed-size arrays as references, improving deserialization ergonomics.
Codec and registry I/O improvements
crates/vm/src/codecs.rs, crates/vm/src/stdlib/io.rs, crates/vm/src/stdlib/winreg.rs
Replaced explicit unwraps and manual slicing with first_chunk() and chunk-based byte extraction in surrogate pass decoding, TextIO cookie parsing, and registry value extraction.
VM utilities refactoring
crates/vm/src/vm/mod.rs
Refactored with_recursion to use ScopeGuard defer for decrement operation, ensuring proper cleanup on all code paths including early returns.
String utilities optimization
crates/wtf8/src/core_str_count.rs
Replaced custom slice_as_chunks helpers with standard library as_chunks() method for chunk iteration.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • RustPython version to 3.14 #6718 — Modifies the same crates/compiler-core/src/marshal.rs file; the referenced PR changes CodeFlags serialization/deserialization while this PR adds the read_array API.

Suggested reviewers

  • youknowone
  • ShaharNaveh
  • arihant2math

Poem

🐰 With first_chunk() and guards so keen,
The crates now safer, cleaner seen,
No more unwraps to fear the fall,
Cell updates handle it all!
Refactored grace through Rust's great hall.

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 63.16% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: switching to Cell::update and slice methods (split_first_chunk, split_off) across multiple files.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/compiler-core/src/marshal.rs (1)

120-122: ⚠️ Potential issue | 🟡 Minor

Guard against overflow and panic in read_array.

N as u32 truncates for large N, and unwrap() can panic if the slice length doesn’t match. Prefer a fallible conversion and error mapping in this public API.

🛡️ Suggested fix
-    fn read_array<const N: usize>(&mut self) -> Result<&[u8; N]> {
-        self.read_slice(N as u32).map(|s| s.try_into().unwrap())
-    }
+    fn read_array<const N: usize>(&mut self) -> Result<&[u8; N]> {
+        let n = u32::try_from(N).map_err(|_| MarshalError::Eof)?;
+        let slice = self.read_slice(n)?;
+        slice.try_into().map_err(|_| MarshalError::Eof)
+    }
Based on learnings: Follow Rust best practices for error handling and memory management.
🤖 Fix all issues with AI agents
In `@crates/vm/src/stdlib/ctypes/function.rs`:
- Around line 570-572: The call to buffer.first_chunk::<{ size_of::<usize>()
}>() uses an unqualified size_of and will fail to compile; update that generic
const to use the fully qualified core::mem::size_of (or add a use) so it becomes
buffer.first_chunk::<{ core::mem::size_of::<usize>() }>() and ensure the
surrounding code that returns usize::from_ne_bytes(bytes) remains unchanged.

In `@crates/vm/src/stdlib/io.rs`:
- Around line 2509-2513: Confirm formatting and linting are clean: run cargo fmt
and cargo clippy, and ensure the slice APIs used in get_field (the as_array()
call on buf, the first_chunk() usage, and the BYTE_LEN constant on the type
implementing this method) are correct per MSRV 1.93.0; if clippy flags any style
or safety issues around the macro get_field or the unchecked unwrap() on
first_chunk(), address them (e.g., handle the Option instead of unwrap or add a
clear comment/expect) so the build and lint checks pass.

Comment on lines +570 to 572
if let Some(&bytes) = buffer.first_chunk::<{ size_of::<usize>() }>() {
return Ok(usize::from_ne_bytes(bytes));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Unqualified size_of will cause a compilation error.

size_of is not imported and needs to be fully qualified. The rest of the file uses core::mem::size_of consistently (e.g., lines 138, 145, 157, 502).

🐛 Proposed fix
-        if let Some(&bytes) = buffer.first_chunk::<{ size_of::<usize>() }>() {
+        if let Some(&bytes) = buffer.first_chunk::<{ core::mem::size_of::<usize>() }>() {
             return Ok(usize::from_ne_bytes(bytes));
         }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if let Some(&bytes) = buffer.first_chunk::<{ size_of::<usize>() }>() {
return Ok(usize::from_ne_bytes(bytes));
}
if let Some(&bytes) = buffer.first_chunk::<{ core::mem::size_of::<usize>() }>() {
return Ok(usize::from_ne_bytes(bytes));
}
🤖 Prompt for AI Agents
In `@crates/vm/src/stdlib/ctypes/function.rs` around lines 570 - 572, The call to
buffer.first_chunk::<{ size_of::<usize>() }>() uses an unqualified size_of and
will fail to compile; update that generic const to use the fully qualified
core::mem::size_of (or add a use) so it becomes buffer.first_chunk::<{
core::mem::size_of::<usize>() }>() and ensure the surrounding code that returns
usize::from_ne_bytes(bytes) remains unchanged.

Comment on lines +2509 to +2513
let buf: &[u8; Self::BYTE_LEN] = buf.as_array()?;
macro_rules! get_field {
($t:ty, $off:ident) => {{
<$t>::from_ne_bytes(
buf[Self::$off..][..core::mem::size_of::<$t>()]
.try_into()
.unwrap(),
)
}};
($t:ty, $off:ident) => {
<$t>::from_ne_bytes(*buf[Self::$off..].first_chunk().unwrap())
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

find . -name "Cargo.toml" -type f | head -20

Repository: RustPython/RustPython

Length of output: 639


🏁 Script executed:

cat Cargo.toml | grep -A 5 -B 5 "rust-version\|package.version"

Repository: RustPython/RustPython

Length of output: 700


🏁 Script executed:

find . -name "rust-toolchain*" -type f

Repository: RustPython/RustPython

Length of output: 86


🏁 Script executed:

rg -n "rust-version" --type toml

Repository: RustPython/RustPython

Length of output: 1141


🏁 Script executed:

cat rust-toolchain.toml

Repository: RustPython/RustPython

Length of output: 95


🏁 Script executed:

rg -n "as_array|first_chunk" crates/vm/src/stdlib/io.rs -B 3 -A 3

Repository: RustPython/RustPython

Length of output: 480


🏁 Script executed:

cd crates/vm && cargo clippy --lib 2>&1 | head -100

Repository: RustPython/RustPython

Length of output: 204


🏁 Script executed:

cd crates/vm && cargo fmt -- --check 2>&1 | grep -A 5 "io.rs"

Repository: RustPython/RustPython

Length of output: 47


🌐 Web query:

as_array() first_chunk() Rust stabilization version 1.76 1.77

💡 Result:

  • first_chunk() (and related first_chunk_mut, split_first_chunk, etc.) was stabilized in Rust 1.77.0. [1]
  • as_array() (and as_mut_array()) is not from 1.76/1.77; slice::as_array was stabilized much later, in Rust 1.93.0. [2]

Sources: [1] [2]


MSRV and API compatibility confirmed; ensure linting and formatting checks are clean.

The repository's MSRV (1.93.0) supports both slice APIs: as_array() was stabilized in Rust 1.93.0, and first_chunk() in Rust 1.77.0. No toolchain compatibility issues.

Per coding guidelines, run cargo fmt and cargo clippy to verify formatting and linting are clean before submission.

🤖 Prompt for AI Agents
In `@crates/vm/src/stdlib/io.rs` around lines 2509 - 2513, Confirm formatting and
linting are clean: run cargo fmt and cargo clippy, and ensure the slice APIs
used in get_field (the as_array() call on buf, the first_chunk() usage, and the
BYTE_LEN constant on the type implementing this method) are correct per MSRV
1.93.0; if clippy flags any style or safety issues around the macro get_field or
the unchecked unwrap() on first_chunk(), address them (e.g., handle the Option
instead of unwrap or add a clear comment/expect) so the build and lint checks
pass.

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Could you also check other issues assigned to you? #6746 and #6894

@youknowone youknowone merged commit 5bf13e8 into RustPython:main Feb 3, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants