diff --git a/book/src/kernel/memory/memory_layout.md b/book/src/kernel/memory/memory_layout.md index 312ba41f..f981903f 100644 --- a/book/src/kernel/memory/memory_layout.md +++ b/book/src/kernel/memory/memory_layout.md @@ -9,7 +9,8 @@ This is the structure of the memory for any process running in the system. ## Layout ```txt 0000_0000_0000_0000 .. FFFF_FF7F_FFFF_FFFF - User (15.99~ EB) -FFFF_FF80_0000_0000 .. FFFF_FFFF_7FFF_FFFF - Process specific kernel (510 GB) +FFFF_FF80_0000_0000 .. FFFF_FF82_0000_0000 - Processes kernel stacks (2 GB, 32768, each 256 KB) +FFFF_FF82_0000_0000 .. FFFF_FF82_0000_1000 - Processes kernel stacks bitmap (4 KB) FFFF_FFFF_8000_0000 .. FFFF_FFFF_FFFF_FFFF - Kernel (2 GB) ``` @@ -40,10 +41,34 @@ This is useful for reading structures that are in specific location in physical Its very simple, it will take memory from the `kernel extra` space, and map it to the physical address. -### Process specific kernel layout +### Processes kernel stacks layout +```txt +FFFF_FF80_0000_0000 .. FFFF_FF80_0000_1000 - process kernel stack 0 guard page (4KB) *not mapped by purpose* +FFFF_FF80_0000_1000 .. FFFF_FF80_0004_0000 - process kernel stack 0 (63 * 4KB = 252KB) +FFFF_FF80_0000_1000 .. FFFF_FF80_0004_1000 - process kernel stack 1 guard page (4KB) *not mapped by purpose* +FFFF_FF80_0004_1000 .. FFFF_FF80_0008_0000 - process kernel stack 1 (63 * 4KB = 252KB) +... +``` + +We have capacity to have `32768` kernel stacks, each of size `256KB`, which is `8GB` in total. + +This space is mapped for each process, but each process have its own segment, but it can still +access all the rest of the kernel stacks. + +This allows us to switch to a process from another process (while in kernel mode), without the need +to switch completely to the kernel stack (used by the kernel). + +See below for the previous design, where each process had its own mapped space that other processes can't access. + +The issue that this new design solves is that, we have to be very careful about when and how to change the context (going to user mode or switching to another process), for example, we can't switch to another process from a syscall, we have to switch +first to kernel mode, and then let the scheduler schedule another process. i.e. scheduler only work on kernel-only stack. + +#### [Outdated] Process specific kernel layout ```txt -FFFF_FF80_0000_0000..FFFF_FF80_0000_1000 process kernel stack guard page (4KB) *not mapped by purpose* -FFFF_FF80_0000_1000..FFFF_FF80_0004_1000 process kernel stack (64 * 4KB = 256KB) +FFFF_FF80_0000_0000 .. FFFF_FFFF_7FFF_FFFF - Process specific kernel (510 GB) +--- +FFFF_FF80_0000_0000 .. FFFF_FF80_0000_1000 - process kernel stack guard page (4KB) *not mapped by purpose* +FFFF_FF80_0000_1000 .. FFFF_FF80_0004_1000 - process kernel stack (64 * 4KB = 256KB) ``` This is a space specific to each process, but reside in kernel space. diff --git a/book/src/kernel/processes/index.md b/book/src/kernel/processes/index.md index 280c4a6d..fc3da71c 100644 --- a/book/src/kernel/processes/index.md +++ b/book/src/kernel/processes/index.md @@ -26,16 +26,20 @@ The process structure [`Process`][process_structure] contain all the information - `priority`: The priority of the process, this is used by the scheduler. see [`PriorityLevel`](https://docs.rs/emerald_kernel_user_link/latest/emerald_kernel_user_link/process/enum.PriorityLevel.html).) - `exit_code`: The exit code of the process, if the process is exited, this will be set to the exit code. - `children_exits`: A list of the children processes that have exited, with their exit code (see #process-exit later for more information). +- `process_kernel_stack`: The kernel stack identifier of the process, this is a 252KB stack that is used when the process is in kernel mode (i.e. when an interrupt happens while the process is in user mode). This stack is shared between all processes, but each process has its own segment to use. ## Process Creation Process creation (structure creation) is as follows: +- Creates a new `VirtualMemoryMapper` instance, which is a clone of the current kernel's virtual memory. +- Allocates new `ProcessKernelStack` which is a 252KB stack for the process to use when + its in kernel mode (interrupts, syscalls, etc...), this memory is mapped and accessible by all + processes, but each process has its own stack segment to use. - Load the `ELF` file, this doesn't load the whole thing, just the header to make sure its valid. - Maps the stack region. - Loads argv into the stack (check [argv structure](#argv-structure) for more information). - Load `ELF` regions into memory. - Load the `Process Metadata` structure (check [Process Metadata](#process-metadata-structure) for more information). -- Add process-specific kernel memory regions, like the kernel stack (**this must be done after loading the ELF, and last modification to the VM manually, because we can't switch to this VM after this point unless its by the scheduler, see the comments `process/mod.rs::allocate_process` for more details**) - Add data about the heap, with size `0` and max size `1GB`, i.e. no memory allocated yet. - Default `context` is created, everything is `0`, except for: - `rip`: The entry point of the `ELF` file. diff --git a/book/src/kernel/processes/scheduler.md b/book/src/kernel/processes/scheduler.md index 56ee0172..6e6b508b 100644 --- a/book/src/kernel/processes/scheduler.md +++ b/book/src/kernel/processes/scheduler.md @@ -23,6 +23,7 @@ here, since we can't do it while the process is running (still handling the `exi Running the process is simple: - copy the `context` of the `process` to the saved `context` of the `CPU`, see [processor saved state](../processor/index.md#saved-cpu-state), which will be used by the [scheduler interrupt](#scheduler-interrupt) to jump to it. - Set the `pid` of the `process` to the `process_id` of the `CPU`. +- Load the `process kernel stack` into the `RSP[KERNEL_RING]` of the `TSS`, this is the stack that will be used when the process is in kernel mode, i.e. when an interrupt happens while the process is in user mode. (see [TSS](../processor/gdt.md#task-state-segment-tss)). - Mark the `process` as `ProcessState::Running`, and move it to the `running_and_waiting` list as mentioned. ## Yielding diff --git a/book/src/kernel/processor/gdt.md b/book/src/kernel/processor/gdt.md index 0c91c736..091c245f 100644 --- a/book/src/kernel/processor/gdt.md +++ b/book/src/kernel/processor/gdt.md @@ -62,8 +62,8 @@ A value of `None` means to use the default stack. The default stack will be the current stack if the privilege level is the same as the current privilege level, otherwise it will change to the stack specified in the [TSS] based on the target privilege level. -Currently, we only have 1 stack for `KERNEL_RING`, which is at `Process kernel stack` in the [memory layout](../memory/memory_layout.md). -I.e. this is a stack specific to each process, as this will only be used when transitioning from user to kernel mode, and inside user mode, we will always be inside a process. +For the `RSP` values, we only use the `KERNEL_RING`, and its being set to the `ProcessKernelStack` of a process before switching to it, i.e. it will change for each process. +This stack is one of the `Processes kernel stacks` (see [memory layout](../memory/memory_layout.md)). [IDT]: https://wiki.osdev.org/Interrupt_Descriptor_Table diff --git a/kernel/src/acpi/mod.rs b/kernel/src/acpi/mod.rs index 1076b716..68767fbb 100644 --- a/kernel/src/acpi/mod.rs +++ b/kernel/src/acpi/mod.rs @@ -123,7 +123,7 @@ impl Acpi { apic::assign_io_irq( acpi_handler as BasicInterruptHandler, facp.sci_interrupt(), - cpu::cpu(), + &cpu::cpu(), ); if !facp.is_acpi_enabled() { diff --git a/kernel/src/cpu/gdt.rs b/kernel/src/cpu/gdt.rs index f2a06509..81db242d 100644 --- a/kernel/src/cpu/gdt.rs +++ b/kernel/src/cpu/gdt.rs @@ -1,20 +1,16 @@ -use core::{mem, ptr::addr_of}; +use core::{mem, pin::Pin, ptr::addr_of}; use crate::{ + cpu, memory_management::{ memory_layout::{ is_aligned, INTR_STACK_BASE, INTR_STACK_EMPTY_SIZE, INTR_STACK_ENTRY_SIZE, - INTR_STACK_SIZE, INTR_STACK_TOTAL_SIZE, PAGE_4K, PROCESS_KERNEL_STACK_END, + INTR_STACK_SIZE, INTR_STACK_TOTAL_SIZE, PAGE_4K, }, - virtual_memory_mapper::{self, VirtualMemoryMapEntry}, + virtual_memory_mapper::{self, ProcessKernelStack, VirtualMemoryMapEntry}, }, - sync::spin::mutex::Mutex, }; -static GDT: Mutex = Mutex::new(GlobalDescriptorManager::empty()); -/// SAFETY: TSS is only used when `GDT` is locked, so its safe to use as `static mut` -static mut TSS: TaskStateSegment = TaskStateSegment::empty(); - pub const KERNEL_RING: u8 = 0; pub const USER_RING: u8 = 3; @@ -29,108 +25,18 @@ impl SegmentSelector { } } -/// This should be called only once, otherwise, it will crash -pub fn init_kernel_gdt() { - let mut manager = GDT.lock(); - if manager.gdt.index != 1 { - panic!("GDT already initialized"); - } - - manager.kernel_code_seg = SegmentSelector::from_index(unsafe { - manager.gdt.push_user(UserDescriptorEntry { - access: flags::PRESENT | flags::CODE | flags::USER | flags::dpl(KERNEL_RING), - flags_and_limit: flags::LONG_MODE, - ..UserDescriptorEntry::empty() - }) - }); - manager.user_code_seg = SegmentSelector::from_index(unsafe { - manager.gdt.push_user(UserDescriptorEntry { - access: flags::PRESENT | flags::CODE | flags::USER | flags::dpl(USER_RING), - flags_and_limit: flags::LONG_MODE, - ..UserDescriptorEntry::empty() - }) - }); - manager.kernel_data_seg = SegmentSelector::from_index(unsafe { - manager.gdt.push_user(UserDescriptorEntry { - access: flags::PRESENT | flags::USER | flags::WRITE | flags::dpl(KERNEL_RING), - ..UserDescriptorEntry::empty() - }) - }); - manager.user_data_seg = SegmentSelector::from_index(unsafe { - manager.gdt.push_user(UserDescriptorEntry { - access: flags::PRESENT | flags::USER | flags::WRITE | flags::dpl(USER_RING), - ..UserDescriptorEntry::empty() - }) - }); - - // setup TSS - - // setup stacks, for each use `INTR_STACK_SIZE` bytes, but also allocate another one of these - // and use as padding between the stacks, so that we can detect stack overflows - for i in 0..7 { - unsafe { - // allocate after an empty offset, so that we can detect stack overflows - let stack_start_virtual = - INTR_STACK_BASE + (i * INTR_STACK_ENTRY_SIZE) + INTR_STACK_EMPTY_SIZE; - let stack_end_virtual = stack_start_virtual + INTR_STACK_SIZE; - assert!(stack_end_virtual <= INTR_STACK_BASE + INTR_STACK_TOTAL_SIZE); - if i == 6 { - // make sure we have allocated everything - assert_eq!(stack_end_virtual, INTR_STACK_BASE + INTR_STACK_TOTAL_SIZE); - } - // make sure that the stack is aligned, so we can easily allocate pages - assert!( - is_aligned(INTR_STACK_SIZE, PAGE_4K) && is_aligned(stack_start_virtual, PAGE_4K) - ); - - // map the stack - virtual_memory_mapper::map_kernel(&VirtualMemoryMapEntry { - virtual_address: stack_start_virtual, - physical_address: None, - size: INTR_STACK_SIZE, - flags: virtual_memory_mapper::flags::PTE_WRITABLE, - }); - - // set the stack pointer - // subtract 8, since the boundary is not mapped - TSS.ist[i] = stack_end_virtual as u64 - 8; - } - - // A kernel stack for this process - // this will be used on transitions from user to kernel - unsafe { TSS.rsp[KERNEL_RING as usize] = PROCESS_KERNEL_STACK_END as u64 - 8 }; - } - - let tss_ptr = addr_of!(TSS) as u64; - - manager.tss_seg = SegmentSelector::from_index(unsafe { - manager.gdt.push_system(SystemDescriptorEntry { - limit: (mem::size_of::() - 1) as u16, - access: flags::PRESENT | flags::TSS_TYPE, - base_low: (tss_ptr & 0xFFFF) as u16, - base_middle: ((tss_ptr >> 16) & 0xFF) as u8, - base_high: ((tss_ptr >> 24) & 0xFF) as u8, - base_upper: ((tss_ptr >> 32) & 0xFFFFFFFF) as u32, - ..SystemDescriptorEntry::empty() - }) - }); - drop(manager); - // call the special `run_with` so that we get the `static` lifetime - GDT.run_with(|manager| { - manager.gdt.apply_lgdt(); - - manager.load_kernel_segments(); - manager.load_tss(); - }); -} - pub fn get_user_code_seg_index() -> SegmentSelector { - GDT.run_with(|manager| manager.user_code_seg) + // TODO: for now, we use the segment from the current CPU + // technically, its all would be the same + cpu::cpu().gdt.user_code_seg } pub fn get_user_data_seg_index() -> SegmentSelector { - GDT.run_with(|manager| manager.user_data_seg) + // TODO: for now, we use the segment from the current CPU + // technically, its all would be the same + cpu::cpu().gdt.user_data_seg } + mod flags { // this is in the flags byte pub const LONG_MODE: u8 = 1 << 5; @@ -208,7 +114,8 @@ impl SystemDescriptorEntry { /// /// This is the structure that is pointed to by the `TSS` descriptor #[repr(C, packed(4))] -struct TaskStateSegment { +#[derive(Debug, Clone, Copy)] +pub struct TaskStateSegment { reserved: u32, rsp: [u64; 3], reserved2: u64, @@ -238,8 +145,10 @@ pub(super) struct GlobalDescriptorTablePointer { base: *const GlobalDescriptorTable, } -struct GlobalDescriptorManager { +#[derive(Debug, Clone, Copy)] +pub struct GlobalDescriptorManager { gdt: GlobalDescriptorTable, + tss: TaskStateSegment, kernel_code_seg: SegmentSelector, user_code_seg: SegmentSelector, // there is only one data segment and its not even used, as we are using @@ -253,6 +162,7 @@ impl GlobalDescriptorManager { pub const fn empty() -> Self { Self { gdt: GlobalDescriptorTable::empty(), + tss: TaskStateSegment::empty(), kernel_code_seg: SegmentSelector::from_index(0), kernel_data_seg: SegmentSelector::from_index(0), user_code_seg: SegmentSelector::from_index(0), @@ -261,6 +171,105 @@ impl GlobalDescriptorManager { } } + fn gdt(self: Pin<&Self>) -> Pin<&GlobalDescriptorTable> { + // SAFETY: This is safe because we are using `Pin<&Self>` which guarantees that + // the GDT is not moved, and we are accessing it in a read-only manner. + unsafe { self.map_unchecked(|s| &s.gdt) } + } + + pub fn init_segments(mut self: Pin<&'static mut Self>) { + if self.gdt.index != 1 { + panic!("GDT already initialized"); + } + + self.kernel_code_seg = SegmentSelector::from_index(unsafe { + self.gdt.push_user(UserDescriptorEntry { + access: flags::PRESENT | flags::CODE | flags::USER | flags::dpl(KERNEL_RING), + flags_and_limit: flags::LONG_MODE, + ..UserDescriptorEntry::empty() + }) + }); + self.user_code_seg = SegmentSelector::from_index(unsafe { + self.gdt.push_user(UserDescriptorEntry { + access: flags::PRESENT | flags::CODE | flags::USER | flags::dpl(USER_RING), + flags_and_limit: flags::LONG_MODE, + ..UserDescriptorEntry::empty() + }) + }); + self.kernel_data_seg = SegmentSelector::from_index(unsafe { + self.gdt.push_user(UserDescriptorEntry { + access: flags::PRESENT | flags::USER | flags::WRITE | flags::dpl(KERNEL_RING), + ..UserDescriptorEntry::empty() + }) + }); + self.user_data_seg = SegmentSelector::from_index(unsafe { + self.gdt.push_user(UserDescriptorEntry { + access: flags::PRESENT | flags::USER | flags::WRITE | flags::dpl(USER_RING), + ..UserDescriptorEntry::empty() + }) + }); + + // setup TSS + + // setup stacks, for each use `INTR_STACK_SIZE` bytes, but also allocate another one of these + // and use as padding between the stacks, so that we can detect stack overflows + for i in 0..7 { + // allocate after an empty offset, so that we can detect stack overflows + let stack_start_virtual = + INTR_STACK_BASE + (i * INTR_STACK_ENTRY_SIZE) + INTR_STACK_EMPTY_SIZE; + let stack_end_virtual = stack_start_virtual + INTR_STACK_SIZE; + assert!(stack_end_virtual <= INTR_STACK_BASE + INTR_STACK_TOTAL_SIZE); + if i == 6 { + // make sure we have allocated everything + assert_eq!(stack_end_virtual, INTR_STACK_BASE + INTR_STACK_TOTAL_SIZE); + } + // make sure that the stack is aligned, so we can easily allocate pages + assert!( + is_aligned(INTR_STACK_SIZE, PAGE_4K) && is_aligned(stack_start_virtual, PAGE_4K) + ); + + // map the stack + virtual_memory_mapper::map_kernel(&VirtualMemoryMapEntry { + virtual_address: stack_start_virtual, + physical_address: None, + size: INTR_STACK_SIZE, + flags: virtual_memory_mapper::flags::PTE_WRITABLE, + }); + + // set the stack pointer + // subtract 8, since the boundary is not mapped + self.tss.ist[i] = stack_end_virtual as u64 - 8; + + // // A kernel stack for this process + // // this will be used on transitions from user to kernel + // self.tss.rsp[KERNEL_RING as usize] = PROCESS_KERNEL_STACK_END as u64 - 8; + } + + let tss_ptr = addr_of!(self.tss) as u64; + + self.tss_seg = SegmentSelector::from_index(unsafe { + self.gdt.push_system(SystemDescriptorEntry { + limit: (mem::size_of::() - 1) as u16, + access: flags::PRESENT | flags::TSS_TYPE, + base_low: (tss_ptr & 0xFFFF) as u16, + base_middle: ((tss_ptr >> 16) & 0xFF) as u8, + base_high: ((tss_ptr >> 24) & 0xFF) as u8, + base_upper: ((tss_ptr >> 32) & 0xFFFFFFFF) as u32, + ..SystemDescriptorEntry::empty() + }) + }); + // convert to ref at this point and do the actual loading + let s = self.into_ref(); + s.gdt().apply_lgdt(); // apply the GDT + s.load_kernel_segments(); + s.load_tss(); + } + + pub fn load_process_kernel_stack(&mut self, stack: &ProcessKernelStack) { + // set the process kernel stack in the TSS + self.tss.rsp[KERNEL_RING as usize] = stack.end_address() as u64 - 8; + } + pub fn load_kernel_segments(&self) { assert_ne!(self.kernel_code_seg.0, 0); unsafe { @@ -281,6 +290,7 @@ impl GlobalDescriptorManager { } #[repr(C, packed(16))] +#[derive(Debug, Clone, Copy)] struct GlobalDescriptorTable { data: [u64; 8], index: usize, @@ -316,11 +326,12 @@ impl GlobalDescriptorTable { index } - pub fn apply_lgdt(&'static self) { + pub fn apply_lgdt(self: Pin<&'static Self>) { let size_used = self.index * mem::size_of::() - 1; + let base: &'static Self = self.get_ref(); let gdt_ptr = GlobalDescriptorTablePointer { limit: size_used as u16, - base: self, + base, }; unsafe { diff --git a/kernel/src/cpu/mod.rs b/kernel/src/cpu/mod.rs index fe20855a..2490a37a 100644 --- a/kernel/src/cpu/mod.rs +++ b/kernel/src/cpu/mod.rs @@ -1,4 +1,9 @@ -use crate::process::ProcessContext; +use core::pin::Pin; + +use crate::{ + cpu::gdt::GlobalDescriptorManager, + memory_management::virtual_memory_mapper::ProcessKernelStack, process::ProcessContext, +}; use self::{ gdt::{GlobalDescriptorTablePointer, SegmentSelector}, @@ -73,6 +78,9 @@ pub struct Cpu { // the process id of the current process pub process_id: u64, pub scheduling: bool, + + /// GDT + gdt: GlobalDescriptorManager, } impl Cpu { @@ -85,6 +93,7 @@ impl Cpu { context: None, process_id: 0, scheduling: false, + gdt: GlobalDescriptorManager::empty(), } } @@ -125,11 +134,25 @@ impl Cpu { pub fn n_cli(&self) -> usize { self.n_cli } + + fn gdt(self: Pin<&'static mut Self>) -> Pin<&'static mut GlobalDescriptorManager> { + // SAFETY: we are guaranteed that `self` is static and never changes + unsafe { self.map_unchecked_mut(|s| &mut s.gdt) } + } + + pub fn init_kernel_gdt(self: Pin<&'static mut Self>) { + // initialize the GDT + self.gdt().init_segments(); + } + + pub fn load_process_kernel_stack(&mut self, stack: &ProcessKernelStack) { + self.gdt.load_process_kernel_stack(stack); + } } -pub fn cpu() -> &'static mut Cpu { +pub fn cpu() -> Pin<&'static mut Cpu> { // TODO: use thread local to get the current cpu - unsafe { &mut CPUS[0] } + Pin::static_mut(unsafe { &mut CPUS[0] }) } pub unsafe fn rflags() -> u64 { diff --git a/kernel/src/devices/clock/hardware_timer/hpet.rs b/kernel/src/devices/clock/hardware_timer/hpet.rs index 74ac817f..df6a4882 100644 --- a/kernel/src/devices/clock/hardware_timer/hpet.rs +++ b/kernel/src/devices/clock/hardware_timer/hpet.rs @@ -237,7 +237,7 @@ impl Hpet { apic::assign_io_irq( timer0_handler as InterruptHandlerWithAllState, chosen_route, - cpu::cpu(), + &cpu::cpu(), ); s.set_enabled(true); diff --git a/kernel/src/devices/clock/hardware_timer/pit.rs b/kernel/src/devices/clock/hardware_timer/pit.rs index a5d2d13e..f1d8b6c1 100644 --- a/kernel/src/devices/clock/hardware_timer/pit.rs +++ b/kernel/src/devices/clock/hardware_timer/pit.rs @@ -131,7 +131,7 @@ impl Pit { apic::assign_io_irq( pit_interrupt as BasicInterruptHandler, pit_io::DEFAULT_INTERRUPT, - cpu::cpu(), + &cpu::cpu(), ); Pit { diff --git a/kernel/src/devices/ide.rs b/kernel/src/devices/ide.rs index cfe1b8a2..d2326b19 100644 --- a/kernel/src/devices/ide.rs +++ b/kernel/src/devices/ide.rs @@ -1085,12 +1085,12 @@ impl PciDevice for IdeDevice { apic::assign_io_irq( ide_interrupt_primary as BasicInterruptHandler, pci_cfg::DEFAULT_PRIMARY_INTERRUPT, - cpu::cpu(), + &cpu::cpu(), ); apic::assign_io_irq( ide_interrupt_secondary as BasicInterruptHandler, pci_cfg::DEFAULT_SECONDARY_INTERRUPT, - cpu::cpu(), + &cpu::cpu(), ); } diff --git a/kernel/src/devices/keyboard_mouse/mod.rs b/kernel/src/devices/keyboard_mouse/mod.rs index 96390128..1368ed5e 100644 --- a/kernel/src/devices/keyboard_mouse/mod.rs +++ b/kernel/src/devices/keyboard_mouse/mod.rs @@ -39,12 +39,12 @@ pub fn init_device() { apic::assign_io_irq( ps2_interrupt_handler as BasicInterruptHandler, KEYBOARD_INT_NUM, - cpu::cpu(), + &cpu::cpu(), ); apic::assign_io_irq( ps2_interrupt_handler as BasicInterruptHandler, MOUSE_INT_NUM, - cpu::cpu(), + &cpu::cpu(), ); } diff --git a/kernel/src/devices/net/e1000.rs b/kernel/src/devices/net/e1000.rs index 68ddf200..d8a1155e 100644 --- a/kernel/src/devices/net/e1000.rs +++ b/kernel/src/devices/net/e1000.rs @@ -492,7 +492,7 @@ pub fn try_register(pci_device: &PciDeviceConfig) -> bool { apic::assign_io_irq( interrupt as BasicInterruptHandler, pci_device.interrupt_line, - cpu::cpu(), + &cpu::cpu(), ); let e1000 = E1000.get().lock(); diff --git a/kernel/src/executable/mod.rs b/kernel/src/executable/mod.rs index 6aa30f34..f5c0a504 100644 --- a/kernel/src/executable/mod.rs +++ b/kernel/src/executable/mod.rs @@ -7,8 +7,7 @@ pub mod elf; /// # Safety /// The `vm` passed must be an exact kernel clone to the current vm -/// without loading new process specific mappings -pub unsafe fn load_elf_to_vm( +pub fn load_elf_to_vm( elf: &elf::Elf, file: &mut fs::File, process_meta: &mut ProcessMetadata, @@ -21,7 +20,7 @@ pub unsafe fn load_elf_to_vm( // switch temporarily so we can map the elf // SAFETY: this must be called while the current vm and this new vm must share the same // kernel regions - vm.switch_to_this(); + unsafe { vm.switch_to_this() }; let mut min_address = usize::MAX; let mut max_address = 0; @@ -85,7 +84,7 @@ pub unsafe fn load_elf_to_vm( } // switch back to the old vm - old_vm.switch_to_this(); + unsafe { old_vm.switch_to_this() }; // we can be interrupted again cpu::cpu().pop_cli(); diff --git a/kernel/src/main.rs b/kernel/src/main.rs index de3d7c4d..05877f39 100644 --- a/kernel/src/main.rs +++ b/kernel/src/main.rs @@ -41,10 +41,7 @@ mod testing; mod utils; use alloc::vec::Vec; -use cpu::{ - gdt, - interrupts::{self, apic}, -}; +use cpu::interrupts::{self, apic}; use executable::elf::Elf; use increasing_heap_allocator::HeapStats; use io::console; @@ -152,7 +149,8 @@ pub extern "C" fn kernel_main(multiboot_info: &'static MultiBoot2Info) -> ! { // require heap allocation console::tracing::move_to_dynamic_buffer(); // must be called before interrupts - gdt::init_kernel_gdt(); + // -- the current/first CPU gdt + cpu::cpu().init_kernel_gdt(); interrupts::init_interrupts(); // mount devices map before initializing them devices::init_devices_mapping(); diff --git a/kernel/src/memory_management/memory_layout.rs b/kernel/src/memory_management/memory_layout.rs index ffd77583..328f6297 100644 --- a/kernel/src/memory_management/memory_layout.rs +++ b/kernel/src/memory_management/memory_layout.rs @@ -48,17 +48,31 @@ pub const KERNEL_EXTRA_MEMORY_BASE: usize = INTR_STACK_BASE + INTR_STACK_TOTAL_S pub const KERNEL_LAST_POSSIBLE_ADDR: usize = 0xFFFF_FFFF_FFFF_F000; pub const KERNEL_EXTRA_MEMORY_SIZE: usize = KERNEL_LAST_POSSIBLE_ADDR - KERNEL_EXTRA_MEMORY_BASE; -// Kernel Data specific to each process (will be mapped differently for each process) -pub const KERNEL_PROCESS_VIRTUAL_ADDRESS_START: usize = - virtual_memory_mapper::KERNEL_PROCESS_VIRTUAL_ADDRESS_START; -pub const PROCESS_KERNEL_STACK_GUARD: usize = PAGE_4K; -// process specific kernel stack, this will be where the process is running while in the kernel -// the process can be interrupted while in the kernel, so we want to save it into a specific stack -// space so that other processes don't override it when being run -pub const PROCESS_KERNEL_STACK_BASE: usize = - KERNEL_PROCESS_VIRTUAL_ADDRESS_START + PROCESS_KERNEL_STACK_GUARD; -pub const PROCESS_KERNEL_STACK_SIZE: usize = PAGE_4K * 64; -pub const PROCESS_KERNEL_STACK_END: usize = PROCESS_KERNEL_STACK_BASE + PROCESS_KERNEL_STACK_SIZE; +// Kernel stacks for userspace processes, each process will allocate and take a segment of these +// these belong to kernel space, and all process will have a copy of these mappings +pub const PROCESSES_KERNEL_STACKS_START: usize = + virtual_memory_mapper::PROCESSES_KERNEL_STACKS_START; +// each of the stacks is 63 pages, and the first page is used for guard page, i.e. not mapped +pub const PROCESSES_KERNEL_STACKS_SIZE_TOTAL: usize = PAGE_4K * 64; +pub const PROCESSES_KERNEL_STACKS_SIZE_MAPPED: usize = PAGE_4K * 63; +pub const MAX_PROCESSES_KERNEL_STACKS: usize = 4096 * 8; // max number of concurrent processes, 1 page of bitmaps +pub const PROCESSES_KERNEL_STACKS_SIZE: usize = + PROCESSES_KERNEL_STACKS_SIZE_TOTAL * MAX_PROCESSES_KERNEL_STACKS; +pub const PROCESSES_KERNEL_STACKS_END: usize = + PROCESSES_KERNEL_STACKS_START + PROCESSES_KERNEL_STACKS_SIZE; +// After the last stack +pub const PROCESSES_KERNEL_STACKS_USED_BITMAP_ADDRESS: usize = PROCESSES_KERNEL_STACKS_END; +pub const PROCESSES_KERNEL_STACKS_USED_BITMAP_SIZE: usize = MAX_PROCESSES_KERNEL_STACKS / 8; // 1 bit per stack, 4096 stacks, so 512 bytes + +/// Returns the start address of the kernel stack for a process at the given index. +pub const fn processes_kernel_stack_start(index: usize) -> usize { + PROCESSES_KERNEL_STACKS_START + index * PROCESSES_KERNEL_STACKS_SIZE_TOTAL + PAGE_4K +} +/// Returns the end address of the kernel stack for a process at the given index. +/// This is the address after the last mapped page of the stack. +pub const fn processes_kernel_stack_end(index: usize) -> usize { + processes_kernel_stack_start(index) + PROCESSES_KERNEL_STACKS_SIZE_MAPPED +} #[allow(dead_code)] pub const KB: usize = 0x400; diff --git a/kernel/src/memory_management/virtual_memory_mapper.rs b/kernel/src/memory_management/virtual_memory_mapper.rs index 7775430a..c4cb84af 100644 --- a/kernel/src/memory_management/virtual_memory_mapper.rs +++ b/kernel/src/memory_management/virtual_memory_mapper.rs @@ -9,18 +9,17 @@ use crate::{ cpu, memory_management::{ memory_layout::{ - align_range, align_up, is_aligned, kernel_elf_rodata_end, physical2virtual, + self, align_range, align_up, is_aligned, kernel_elf_rodata_end, physical2virtual, virtual2physical, MemSize, EXTENDED_OFFSET, KERNEL_BASE, KERNEL_END, KERNEL_LINK, - KERNEL_MAPPED_SIZE, PAGE_2M, PAGE_4K, + KERNEL_MAPPED_SIZE, PAGE_2M, PAGE_4K, PROCESSES_KERNEL_STACKS_USED_BITMAP_ADDRESS, + PROCESSES_KERNEL_STACKS_USED_BITMAP_SIZE, }, physical_page_allocator, }, sync::{once::OnceLock, spin::mutex::Mutex}, }; -use super::memory_layout::{ - stack_guard_page_ptr, PROCESS_KERNEL_STACK_BASE, PROCESS_KERNEL_STACK_SIZE, -}; +use super::memory_layout::stack_guard_page_ptr; // TODO: replace by some sort of bitfield #[allow(dead_code)] @@ -44,16 +43,13 @@ const ADDR_MASK: u64 = 0x0000_0000_FFFF_F000; const KERNEL_L4_INDEX: usize = 0x1FF; // The L3 positions are used for the non-moving kernel code/data -const KERNEL_L3_INDEX_START: usize = 0x1FE; +const KERNEL_L3_INDEX_START: usize = 0; #[allow(dead_code)] const KERNEL_L3_INDEX_END: usize = 0x1FF; -const KERNEL_L3_PROCESS_INDEX_START: usize = 0; -const KERNEL_L3_PROCESS_INDEX_END: usize = KERNEL_L3_INDEX_START - 1; - -pub const KERNEL_PROCESS_VIRTUAL_ADDRESS_START: usize = +pub const PROCESSES_KERNEL_STACKS_START: usize = // sign extension - 0xFFFF_0000_0000_0000 | KERNEL_L4_INDEX << 39 | KERNEL_L3_PROCESS_INDEX_START << 30; + 0xFFFF_0000_0000_0000 | KERNEL_L4_INDEX << 39 | KERNEL_L3_INDEX_START << 30; // the user can use all the indexes except the last one const NUM_USER_L4_INDEXES: usize = KERNEL_L4_INDEX; @@ -162,7 +158,7 @@ impl PageDirectoryTablePtr { } } -static KERNEL_VIRTUAL_MEMORY_MANAGER: OnceLock> = OnceLock::new(); +static KERNEL_VIRTUAL_MEMORY_MANAGER: OnceLock> = OnceLock::new(); pub fn init_kernel_vm() { if KERNEL_VIRTUAL_MEMORY_MANAGER.try_get().is_some() { @@ -170,23 +166,31 @@ pub fn init_kernel_vm() { } let manager = KERNEL_VIRTUAL_MEMORY_MANAGER - .get_or_init(|| Mutex::new(VirtualMemoryMapper::new_kernel_vm())) + .get_or_init(|| Mutex::new(KernelVirtualMemoryManager::new_kernel_vm())) .lock(); // // SAFETY: this is the start VM, so we are sure that we are not inside a process, so its safe to switch - unsafe { manager.switch_to_this() }; + unsafe { manager.kernel_vm.switch_to_this() }; } /// # Safety /// This must never be called while we are in a process context /// and using any process specific memory regions pub unsafe fn switch_to_kernel() { - KERNEL_VIRTUAL_MEMORY_MANAGER.get().lock().switch_to_this(); + KERNEL_VIRTUAL_MEMORY_MANAGER + .get() + .lock() + .kernel_vm + .switch_to_this(); } pub fn map_kernel(entry: &VirtualMemoryMapEntry) { // make sure we are only mapping to kernel memory assert!(entry.virtual_address >= KERNEL_BASE); - KERNEL_VIRTUAL_MEMORY_MANAGER.get().lock().map(entry); + KERNEL_VIRTUAL_MEMORY_MANAGER + .get() + .lock() + .kernel_vm + .map(entry); } /// `is_allocated` is used to indicate if the physical pages were allocated by the caller @@ -199,6 +203,7 @@ pub fn unmap_kernel(entry: &VirtualMemoryMapEntry, is_allocated: bool) { KERNEL_VIRTUAL_MEMORY_MANAGER .get() .lock() + .kernel_vm .unmap(entry, is_allocated); } @@ -207,6 +212,7 @@ pub fn is_address_mapped_in_kernel(addr: usize) -> bool { KERNEL_VIRTUAL_MEMORY_MANAGER .get() .lock() + .kernel_vm .is_address_mapped(addr) } @@ -221,108 +227,67 @@ pub fn clone_current_vm_as_user() -> VirtualMemoryMapper { } pub fn get_current_vm() -> VirtualMemoryMapper { - VirtualMemoryMapper::get_current_vm() + let kernel_vm_addr = KERNEL_VIRTUAL_MEMORY_MANAGER + .get() + .lock() + .kernel_vm + .page_map_l4 + .as_physical(); + let cr3 = unsafe { cpu::get_cr3() }; // cr3 is physical address + let is_user = cr3 != kernel_vm_addr; + VirtualMemoryMapper { + page_map_l4: PageDirectoryTablePtr::from_entry(cr3), + is_user, + } } -pub struct VirtualMemoryMapper { - page_map_l4: PageDirectoryTablePtr, - is_user: bool, -} +/// A process kernel stack, this is just a handler containing the index of the stack. +#[derive(Debug)] +pub struct ProcessKernelStack(u32); -impl VirtualMemoryMapper { - fn new() -> Self { - Self { - page_map_l4: PageDirectoryTablePtr::alloc_new(), - is_user: false, - } +impl ProcessKernelStack { + pub fn allocate() -> Self { + KERNEL_VIRTUAL_MEMORY_MANAGER + .get() + .lock() + .allocate_process_kernel_stack() } - // create a new virtual memory that maps the kernel only - pub fn clone_kernel_mem(&self) -> Self { - let this_kernel_l4 = - PageDirectoryTablePtr::from_entry(self.page_map_l4.as_ref().entries[KERNEL_L4_INDEX]); - - let mut new_vm = Self::new(); - - let mut new_kernel_l4 = PageDirectoryTablePtr::alloc_new(); - - // copy the whole kernel mapping (process specific will be replaced later) - for i in 0..=0x1FF { - new_kernel_l4.as_mut().entries[i] = this_kernel_l4.as_ref().entries[i]; - } - - new_vm.page_map_l4.as_mut().entries[KERNEL_L4_INDEX] = - new_kernel_l4.as_physical() | flags::PTE_PRESENT | flags::PTE_WRITABLE; - - new_vm + pub const fn start_address(&self) -> usize { + memory_layout::processes_kernel_stack_start(self.0 as usize) } - /// # Safety - /// - /// After this call, the VM must never be switched to unless - /// its from the scheduler or we are sure that the previous kernel regions are not used - pub unsafe fn add_process_specific_mappings(&mut self) { - let mut this_kernel_l4 = - PageDirectoryTablePtr::from_entry(self.page_map_l4.as_ref().entries[KERNEL_L4_INDEX]); - - // clear out the process specific mappings if we have cloned another process - // but of course don't deallocate, just remove the mappings - for i in KERNEL_L3_PROCESS_INDEX_START..=KERNEL_L3_PROCESS_INDEX_END { - this_kernel_l4.as_mut().entries[i] = 0; - } - // set it temporarily so we can map kernel range - // TODO: fix this hack - self.is_user = false; - // load new kernel stack for this process - self.map(&VirtualMemoryMapEntry { - virtual_address: PROCESS_KERNEL_STACK_BASE, - physical_address: None, // allocate - size: PROCESS_KERNEL_STACK_SIZE, - flags: flags::PTE_WRITABLE, - }); - self.is_user = true; + pub const fn end_address(&self) -> usize { + memory_layout::processes_kernel_stack_end(self.0 as usize) } - fn load_vm(base: &PageDirectoryTablePtr) { - trace!( - "Switching to new page map: {:p}", - base.as_physical() as *const u8 - ); - unsafe { cpu::set_cr3(base.as_physical()) } + pub const fn size(&self) -> usize { + self.end_address() - self.start_address() } +} - fn get_current_vm() -> Self { - let kernel_vm_addr = KERNEL_VIRTUAL_MEMORY_MANAGER +impl Drop for ProcessKernelStack { + fn drop(&mut self) { + // free the stack when dropped + KERNEL_VIRTUAL_MEMORY_MANAGER .get() .lock() - .page_map_l4 - .as_physical(); - let cr3 = unsafe { cpu::get_cr3() }; // cr3 is physical address - let is_user = cr3 != kernel_vm_addr; - Self { - page_map_l4: PageDirectoryTablePtr::from_entry(cr3), - is_user, - } - } - - /// Return `true` if the current VM is used by the current cpu - pub fn is_used_by_me(&self) -> bool { - let cr3 = unsafe { cpu::get_cr3() }; - cr3 == self.page_map_l4.as_physical() + .free_process_kernel_stack(self); } +} - /// # Safety - /// This must be used with caution, it must never be switched while we are using - /// memory from the same regions, i.e. kernel stack while we are in an interrupt - pub unsafe fn switch_to_this(&self) { - Self::load_vm(&self.page_map_l4); - } +pub struct KernelVirtualMemoryManager { + kernel_vm: VirtualMemoryMapper, + // on each bit, a value of 1 means that the stack is used and allocated + processes_kernel_stacks_bitmap: &'static mut [u8; PROCESSES_KERNEL_STACKS_USED_BITMAP_SIZE], +} +impl KernelVirtualMemoryManager { // This replicate what is done in the assembly code // but it will be stored fn new_kernel_vm() -> Self { let data_start = align_up(kernel_elf_rodata_end(), PAGE_4K); - let kernel_vm = [ + let kernel_vm_entries = [ // Low memory (has some BIOS stuff): mapped to kernel space VirtualMemoryMapEntry { virtual_address: KERNEL_BASE, @@ -349,14 +314,14 @@ impl VirtualMemoryMapper { // create a new fresh page map // SAFETY: we are calling the virtual memory manager after initializing the physical page allocator - let mut s = Self::new(); + let mut kernel_vm = VirtualMemoryMapper::new(); - for entry in kernel_vm.iter() { - s.map(entry); + for entry in kernel_vm_entries.iter() { + kernel_vm.map(entry); } // unmap stack guard - s.unmap( + kernel_vm.unmap( &VirtualMemoryMapEntry { virtual_address: stack_guard_page_ptr(), physical_address: None, @@ -366,7 +331,126 @@ impl VirtualMemoryMapper { false, ); - s + // map the processes kernel stacks bitmap + kernel_vm.map(&VirtualMemoryMapEntry { + virtual_address: PROCESSES_KERNEL_STACKS_USED_BITMAP_ADDRESS, + physical_address: None, + size: PROCESSES_KERNEL_STACKS_USED_BITMAP_SIZE, + flags: flags::PTE_WRITABLE, + }); + + Self { + kernel_vm, + processes_kernel_stacks_bitmap: unsafe { + (PROCESSES_KERNEL_STACKS_USED_BITMAP_ADDRESS as *mut u8 + as *mut [u8; PROCESSES_KERNEL_STACKS_USED_BITMAP_SIZE]) + .as_mut() + .expect("This can't be null") + }, + } + } + + fn allocate_process_kernel_stack(&mut self) -> ProcessKernelStack { + // find the first free stack + let index = self + .processes_kernel_stacks_bitmap + .iter_mut() + .position(|x| *x != 0xFF) + .expect("No free kernel stacks available"); + + for i in 0..8 { + if self.processes_kernel_stacks_bitmap[index] & (1 << i) == 0 { + // set it to used + self.processes_kernel_stacks_bitmap[index] |= 1 << i; + + let stack_index = + ProcessKernelStack((index * 8 + i).try_into().expect("Stack index too large")); + + self.kernel_vm.map(&VirtualMemoryMapEntry { + virtual_address: stack_index.start_address(), + physical_address: None, + size: stack_index.size(), + flags: flags::PTE_WRITABLE | flags::PTE_USER, + }); + + return stack_index; + } + } + + unreachable!("This should never happen, as we checked for free stack above"); + } + + fn free_process_kernel_stack(&mut self, stack: &mut ProcessKernelStack) { + let index = stack.0 as usize / 8; + let bit = stack.0 as usize % 8; + + // clear the bit + self.processes_kernel_stacks_bitmap[index] &= !(1 << bit); + + // unmap the stack + self.kernel_vm.unmap( + &VirtualMemoryMapEntry { + virtual_address: stack.start_address(), + physical_address: None, + size: stack.size(), + flags: flags::PTE_WRITABLE | flags::PTE_USER, + }, + false, + ); + } +} + +pub struct VirtualMemoryMapper { + page_map_l4: PageDirectoryTablePtr, + is_user: bool, +} + +impl VirtualMemoryMapper { + fn new() -> Self { + Self { + page_map_l4: PageDirectoryTablePtr::alloc_new(), + is_user: false, + } + } + + // create a new virtual memory that maps the kernel only + pub fn clone_kernel_mem(&self) -> Self { + let mut new_vm = Self::new(); + + assert_ne!( + self.page_map_l4.as_ref().entries[KERNEL_L4_INDEX] & flags::PTE_PRESENT, + 0 + ); + assert_ne!( + self.page_map_l4.as_ref().entries[KERNEL_L4_INDEX] & flags::PTE_WRITABLE, + 0 + ); + + new_vm.page_map_l4.as_mut().entries[KERNEL_L4_INDEX] = + self.page_map_l4.as_ref().entries[KERNEL_L4_INDEX]; + + new_vm + } + + fn load_vm(base: &PageDirectoryTablePtr) { + trace!( + "Switching to new page map: {:p}", + base.as_physical() as *const u8 + ); + unsafe { cpu::set_cr3(base.as_physical()) } + } + + /// Return `true` if the current VM is used by the current cpu + pub fn is_used_by_me(&self) -> bool { + let cr3 = unsafe { cpu::get_cr3() }; + cr3 == self.page_map_l4.as_physical() + } + + /// # Safety + /// This must be used with caution, it must never be switched while we are using + /// memory from the same regions, references and pointers might be invalidated + pub unsafe fn switch_to_this(&self) { + Self::load_vm(&self.page_map_l4); } pub fn map(&mut self, entry: &VirtualMemoryMapEntry) { @@ -813,15 +897,6 @@ impl VirtualMemoryMapper { self.do_for_ranges_entries(0..NUM_USER_L4_INDEXES, 0..=0x1FF, f) } - // the handler function definition is `fn(page_entry: &mut u64)` - fn do_for_kernel_process_entry(&mut self, f: impl FnMut(&mut u64)) { - self.do_for_ranges_entries( - KERNEL_L4_INDEX..=KERNEL_L4_INDEX, - KERNEL_L3_PROCESS_INDEX_START..=KERNEL_L3_PROCESS_INDEX_END, - f, - ); - } - // search for all the pages that are mapped to the user ranges and unmap them and free their memory // also unmap any process specific kernel memory pub fn unmap_process_memory(&mut self) { @@ -837,6 +912,5 @@ impl VirtualMemoryMapper { }; self.do_for_every_user_entry(free_page); - self.do_for_kernel_process_entry(free_page); } } diff --git a/kernel/src/process/mod.rs b/kernel/src/process/mod.rs index 4e773ca9..8ac37b66 100644 --- a/kernel/src/process/mod.rs +++ b/kernel/src/process/mod.rs @@ -17,7 +17,8 @@ use crate::{ memory_management::{ memory_layout::{align_down, align_up, is_aligned, GB, KERNEL_BASE, MB, PAGE_2M, PAGE_4K}, virtual_memory_mapper::{ - self, VirtualMemoryMapEntry, VirtualMemoryMapper, MAX_USER_VIRTUAL_ADDRESS, + self, ProcessKernelStack, VirtualMemoryMapEntry, VirtualMemoryMapper, + MAX_USER_VIRTUAL_ADDRESS, }, }, }; @@ -128,6 +129,8 @@ pub struct Process { // split from the state, so that we can keep it as a simple enum exit_code: i32, children_exits: BTreeMap, + + process_kernel_stack: ProcessKernelStack, } impl Process { @@ -141,6 +144,8 @@ impl Process { let id = PROCESS_ID_ALLOCATOR.allocate(); let mut vm = virtual_memory_mapper::clone_current_vm_as_user(); + let process_kernel_stack = ProcessKernelStack::allocate(); + let mut process_meta = ProcessMetadata::empty(); process_meta.pid = id; let process_meta_addr = MAX_USER_VIRTUAL_ADDRESS - PAGE_4K; @@ -168,16 +173,11 @@ impl Process { let (new_rsp, argc, argv_ptr) = Self::prepare_stack(&mut vm, &argv, rsp, stack_start as u64); - // SAFETY: we know that the vm passed is an exact kernel copy of this vm, so its safe to switch to it // TODO: maybe it would be best to create the new vm inside this function? - let (_min_addr, max_addr) = - unsafe { load_elf_to_vm(elf, file, &mut process_meta, &mut vm)? }; + let (_min_addr, max_addr) = load_elf_to_vm(elf, file, &mut process_meta, &mut vm)?; Self::write_process_meta(&mut vm, process_meta_addr, process_meta); - // SAFETY: we know that the vm is never used after this point until scheduling - unsafe { vm.add_process_specific_mappings() }; - // set it quite a distance from the elf and align it to 2MB pages (we are not using 2MB virtual memory, so its not related) let heap_start = align_up(max_addr + HEAP_OFFSET_FROM_ELF_END, PAGE_2M); let heap_size = 0; // start at 0, let user space programs control it @@ -217,6 +217,7 @@ impl Process { priority: PriorityLevel::Normal, exit_code: 0, children_exits: BTreeMap::new(), + process_kernel_stack, }) } diff --git a/kernel/src/process/scheduler.rs b/kernel/src/process/scheduler.rs index 56612161..87609299 100644 --- a/kernel/src/process/scheduler.rs +++ b/kernel/src/process/scheduler.rs @@ -236,7 +236,7 @@ pub fn schedule() { SCHEDULER.lock().init_interrupt(); loop { - let current_cpu = cpu::cpu(); + let mut current_cpu = cpu::cpu(); assert!(current_cpu.context.is_none()); let mut scheduler = SCHEDULER.lock(); @@ -275,12 +275,13 @@ pub fn schedule() { top.priority_counter -= decrement; scheduler.max_priority = top.priority_counter; - // SAFETY: we are the scheduler and running in kernel space, so it's safe to switch to this vm - // as it has clones of our kernel mappings + // SAFETY: we are the scheduler and running in kernel space that is shared by al processes, + // so it's safe to switch to this vm as it has clones of our kernel mappings unsafe { inner_proc.switch_to_this_vm() }; current_cpu.process_id = inner_proc.id; current_cpu.context = Some(inner_proc.context); current_cpu.scheduling = true; + current_cpu.load_process_kernel_stack(&inner_proc.process_kernel_stack); } scheduler.running_waiting_procs.insert(pid, top); } @@ -373,7 +374,7 @@ where /// The caller of this function (i.e. interrupt) will use the `all_state` to go back to the scheduler. /// This function will remove the context from the CPU, and thus the value in `all_state` will be dropped. pub fn exit_current_process(exit_code: i32, all_state: &mut InterruptAllSavedState) { - let current_cpu = cpu::cpu(); + let mut current_cpu = cpu::cpu(); assert!(current_cpu.context.is_some()); current_cpu.push_cli(); @@ -399,7 +400,7 @@ pub fn exit_current_process(exit_code: i32, all_state: &mut InterruptAllSavedSta } pub fn sleep_current_process(time: ClockTime, all_state: &mut InterruptAllSavedState) { - let current_cpu = cpu::cpu(); + let mut current_cpu = cpu::cpu(); assert!(current_cpu.context.is_some()); let deadline = clock::clocks().time_since_startup() + time; @@ -423,7 +424,7 @@ pub fn sleep_current_process(time: ClockTime, all_state: &mut InterruptAllSavedS } pub fn yield_current_if_any(all_state: &mut InterruptAllSavedState) { - let current_cpu = cpu::cpu(); + let mut current_cpu = cpu::cpu(); // do not yield if we don't have context, or we are in the middle of scheduling if current_cpu.context.is_none() || current_cpu.scheduling { return; @@ -455,7 +456,7 @@ pub fn is_process_running(pid: u64) -> bool { } pub fn wait_for_pid(all_state: &mut InterruptAllSavedState, pid: u64) -> bool { - let current_cpu = cpu::cpu(); + let mut current_cpu = cpu::cpu(); assert!(current_cpu.context.is_some()); // we can't wait for a process that doesn't exist now, unless we are a parent of a process that has exited @@ -521,7 +522,7 @@ pub fn swap_context(context: &mut ProcessContext, all_state: &mut InterruptAllSa extern "C" fn scheduler_interrupt_handler(all_state: &mut InterruptAllSavedState) { assert_eq!(all_state.frame.cs & 0x3, 0, "must be from kernel only"); - let current_cpu = cpu::cpu(); + let mut current_cpu = cpu::cpu(); assert!(current_cpu.context.is_some()); assert!(current_cpu.scheduling); assert!(current_cpu.interrupts_disabled()); diff --git a/kernel/src/sync/spin/mutex.rs b/kernel/src/sync/spin/mutex.rs index 8d561068..5bdc1550 100644 --- a/kernel/src/sync/spin/mutex.rs +++ b/kernel/src/sync/spin/mutex.rs @@ -67,7 +67,7 @@ impl Mutex { impl Mutex { pub fn lock(&self) -> MutexGuard<'_, T> { - let cpu = cpu::cpu(); + let mut cpu = cpu::cpu(); cpu.push_cli(); // disable interrupts to avoid deadlock let cpu_id = cpu.id as i64; @@ -84,7 +84,7 @@ impl Mutex { } pub fn try_lock(&self) -> Option> { - let cpu = cpu::cpu(); + let mut cpu = cpu::cpu(); cpu.push_cli(); // disable interrupts to avoid deadlock let cpu_id = cpu.id as i64; diff --git a/kernel/src/sync/spin/remutex.rs b/kernel/src/sync/spin/remutex.rs index 10e11501..8726b45d 100644 --- a/kernel/src/sync/spin/remutex.rs +++ b/kernel/src/sync/spin/remutex.rs @@ -68,7 +68,7 @@ impl ReMutex { } pub fn lock(&self) -> ReMutexGuard<'_, T> { - let cpu = cpu::cpu(); + let mut cpu = cpu::cpu(); cpu.push_cli(); // disable interrupts to avoid deadlock let cpu_id = cpu.id as i64; @@ -97,7 +97,7 @@ impl ReMutex { } pub fn try_lock(&self) -> Option> { - let cpu = cpu::cpu(); + let mut cpu = cpu::cpu(); cpu.push_cli(); // disable interrupts to avoid deadlock let cpu_id = cpu.id as i64; diff --git a/kernel/src/sync/spin/rwlock.rs b/kernel/src/sync/spin/rwlock.rs index 80729ac5..b160ef72 100644 --- a/kernel/src/sync/spin/rwlock.rs +++ b/kernel/src/sync/spin/rwlock.rs @@ -118,7 +118,7 @@ impl RwLock { } pub fn write(&self) -> RwLockWriteGuard<'_, T> { - let cpu = cpu::cpu(); + let mut cpu = cpu::cpu(); cpu.push_cli(); // disable interrupts to avoid deadlock let cpu_id = cpu.id as i64; @@ -135,7 +135,7 @@ impl RwLock { } pub fn try_write(&self) -> Option> { - let cpu = cpu::cpu(); + let mut cpu = cpu::cpu(); cpu.push_cli(); // disable interrupts to avoid deadlock let cpu_id = cpu.id as i64;