Skip to content
charmOS docs

Scheduler

include/sch/sched.h View source View on GitHub

Structs

Section titled “Structs”

idle_thread_data

Section titled “idle_thread_data”
struct idle_thread_data (2 fields) source
§ atomic_uint64_t last_entry_ms
§ uint64_t last_exit_ms

scheduler

Section titled “scheduler”
struct scheduler (32 fields) source

Current tick data

§ atomic_bool tick_enabled
§ time_ms_t tick_duration_ms

Structures

§ struct list_head urgent_threads
§ struct rbt thread_rbt
§ struct rbt completed_rbt
§ struct list_head rt_threads
§ struct list_head bg_threads
§ struct rbt climb_threads

threads on this CPU participating in CLIMB

§ atomic_uint8_t queue_bitmap
§ struct thread *current
§ struct thread *drop_last_ref
§ struct timer tick
§ struct spinlock wroost_lock
§ struct wroost_percpu_data *wroost_data
§ size_t thread_count[THREAD_PRIO_CLASS_COUNT]

Thread count at each prio

§ size_t total_thread_count
§ size_t total_weight

Period information

§ bool period_enabled
§ uint64_t current_period
§ time_ms_t period_ms
§ time_ms_t period_start_ms

Timestamp

§ cpu_id_t core_id

Work steal/migration

§ atomic_bool being_robbed
TODO

Rework time load balancing away from this foolery

§ atomic_bool stealing_work
§ struct spinlock lock

Idle thread data

§ struct thread *idle_thread
§ struct idle_thread_data idle_thread_data
§ struct scheduler *other_locked

If we acquired the lock of another scheduler in scheduler_yield(), we store a pointer to it here.

If this is NULL, we didn’t do that, but in the case that it isn’t, we must drop the raw lock for this in addition to the lock for our scheduler

§ enum irql locked_irql
§ enum irql other_locked_irql
§ struct cpu_mask deferred_resched_mask
§ struct rt_scheduler_percpu *rt

scheduler_data

Section titled “scheduler_data”
struct scheduler_data (4 fields) source

For a global structure containing central scheduler data

§ uint32_t max_concurrent_stealers
§ atomic_uint32_t active_stealers
§ atomic_uint32_t total_threads
§ atomic_int64_t steal_min_diff

Functions

Section titled “Functions”

scheduler_init

Section titled “scheduler_init”
void scheduler_init(void)
source

scheduler_debug_start

Section titled “scheduler_debug_start”
void scheduler_debug_start(void)
source

scheduler_debug_start

Section titled “scheduler_debug_start”
void scheduler_debug_start(void)
source

scheduler_select_best_for_thread

Section titled “scheduler_select_best_for_thread”
struct scheduler * scheduler_select_best_for_thread(struct thread *t)
source

scheduler_add_thread

Section titled “scheduler_add_thread”
void scheduler_add_thread(
    struct scheduler *sched,
    struct thread *thread
)
source

scheduler_remove_thread

Section titled “scheduler_remove_thread”
void scheduler_remove_thread(
    struct scheduler *sched,
    struct thread *t
)
source

schedule

Section titled “schedule”
void schedule(void)
source

k_sch_main

Section titled “k_sch_main”
void k_sch_main(void * *)
source

scheduler_idle_main

Section titled “scheduler_idle_main”
void scheduler_idle_main(void * *)
source

scheduler_yield

Section titled “scheduler_yield”
void scheduler_yield(void)
source

scheduler_period_start

Section titled “scheduler_period_start”
void scheduler_period_start(struct scheduler *s, uint64_t now_ms)
source

switch_context

Section titled “switch_context”
void switch_context(struct cpu_context *old, struct cpu_context *new)
source

load_context

Section titled “load_context”
void load_context(struct cpu_context *new)
source

save_context

Section titled “save_context”
void save_context(struct cpu_context *new)
source

scheduler_can_steal_work

Section titled “scheduler_can_steal_work”
bool scheduler_can_steal_work(struct scheduler *sched)
source

scheduler_can_take_thread

Section titled “scheduler_can_take_thread”
bool scheduler_can_take_thread(size_t core, struct thread *target)
source

scheduler_compute_steal_threshold

Section titled “scheduler_compute_steal_threshold”
uint64_t scheduler_compute_steal_threshold(void)
source

scheduler_try_do_steal

Section titled “scheduler_try_do_steal”
struct thread * scheduler_try_do_steal(struct scheduler *sched)
source

scheduler_pick_victim

Section titled “scheduler_pick_victim”
struct scheduler * scheduler_pick_victim(struct scheduler *self)
source

scheduler_steal_work

Section titled “scheduler_steal_work”
struct thread * scheduler_steal_work(
    struct scheduler *new,
    struct scheduler *victim
)
source

scheduler_try_push_to_idle_core

Section titled “scheduler_try_push_to_idle_core”
size_t scheduler_try_push_to_idle_core(struct scheduler *sched)
source

scheduler_tick_enable

Section titled “scheduler_tick_enable”
void scheduler_tick_enable(void)
source

scheduler_tick_disable

Section titled “scheduler_tick_disable”
void scheduler_tick_disable(void)
source

scheduler_reschedule_isr

Section titled “scheduler_reschedule_isr”
enum irq_result scheduler_reschedule_isr(
    struct irq_context *rsp,
    void *ctx
)
source

scheduler_self_in_resched

Section titled “scheduler_self_in_resched”
bool scheduler_self_in_resched(void)
source

scheduler_mark_self_in_resched

Section titled “scheduler_mark_self_in_resched”
bool scheduler_mark_self_in_resched(bool new)
source

Because this is always executed at or above DISPATCH it is fine

scheduler_mark_core_needs_resched

Section titled “scheduler_mark_core_needs_resched”
bool scheduler_mark_core_needs_resched(struct core *c, bool new)
source

scheduler_mark_self_needs_resched

Section titled “scheduler_mark_self_needs_resched”
bool scheduler_mark_self_needs_resched(bool new)
source

scheduler_mark_core_needs_run_dpcs

Section titled “scheduler_mark_core_needs_run_dpcs”
bool scheduler_mark_core_needs_run_dpcs(struct core *c, bool new)
source

scheduler_mark_self_needs_run_dpcs

Section titled “scheduler_mark_self_needs_run_dpcs”
bool scheduler_mark_self_needs_run_dpcs(bool new)
source

scheduler_mark_self_idle

Section titled “scheduler_mark_self_idle”
void scheduler_mark_self_idle(bool new)
source

this is only ever called when a thread is loaded

scheduler_resched_if_needed

Section titled “scheduler_resched_if_needed”
void scheduler_resched_if_needed(void)
source

scheduler_core_idle

Section titled “scheduler_core_idle”
bool scheduler_core_idle(struct core *c)
source

scheduler_force_resched

Section titled “scheduler_force_resched”
void scheduler_force_resched(struct scheduler *sched)
source
NOTE

this function is unsafe to call from context that holds a runqueue lock. The reason:

Imagine the following:     CPU 0 is holding RQ 0’s lock     CPU 0 IPI’s CPU 1     CPU 1, before that IPI arrives, was spinning on            RQ 0’s lock, since it was migrating a thread.

           Thus, CPU 1 cannot respond to the IPI            because it has interrupts disabled,            and will never enter the ISR to ack.

    CPU 0 spins on waiting for this (uncompletable) IPI to complete     CPU 1 is also stuck waiting for RQ 0’s lock to be relinquished,           which is something that won’t happen because of this           dependency

    boom deadlock.

    scheduler core code is supposed to call scheduler_defer_resched()     defined in the internal header there.

scheduler_try_force_resched

Section titled “scheduler_try_force_resched”
bool cc_warn_unused_result scheduler_try_force_resched(struct scheduler *sched)
source

scheduler_force_run_dpcs

Section titled “scheduler_force_run_dpcs”
void scheduler_force_run_dpcs(cpu_id_t cpu)
source

Uses the blocking ipi_send(), so no spinlock may be held.

scheduler_preemption_disabled

Section titled “scheduler_preemption_disabled”
bool scheduler_preemption_disabled(enum topology_caller c)
source

scheduler_change_tick_duration

Section titled “scheduler_change_tick_duration”
void scheduler_change_tick_duration(uint64_t new_duration)
source

Variables

Section titled “Variables”

scheduler_data

Section titled “scheduler_data”
extern struct scheduler_data scheduler_data
source

Macros

Section titled “Macros”

WORK_STEAL_THRESHOLD

Section titled “WORK_STEAL_THRESHOLD”
#define WORK_STEAL_THRESHOLD
source
Expansion
#define WORK_STEAL_THRESHOLD \
    75ULL /* How little work the core needs to be \
           * doing to try to steal work from another \
           * core. This means "% of the average" \
           */

SCHEDULER_DEFAULT_WORK_STEAL_MIN_DIFF

Section titled “SCHEDULER_DEFAULT_WORK_STEAL_MIN_DIFF”
#define SCHEDULER_DEFAULT_WORK_STEAL_MIN_DIFF 130
source