Skip to content
charmOS docs

Per-CPU Reference Counter

include/sync/percpu_rc.h View source View on GitHub

Per-CPU Reference Counter

Fast, no contention reference counting by distributing RC inc/dec across per-CPU counters in fast path, using large bias and RCU GP to transition to atomic counter without lost updates

Structs

Section titled “Structs”

percpu_rc

Section titled “percpu_rc”
struct percpu_rc (5 fields) source
§ atomic_uintptr_t percpu_count_ptr
§ bool allow_reinit
§ struct rcu_cb rcu

Enums

Section titled “Enums”

percpu_rc_flags

Section titled “percpu_rc_flags”
enum percpu_rc_flags (2 variants) source
§ PERCPU_RC_INIT_ATOMIC = BIT(0)
§ PERCPU_RC_ALLOW_REINIT = BIT(1)

Type Aliases

Section titled “Type Aliases”

percpu_rc_release_fn

Section titled “percpu_rc_release_fn”
typedef void (*percpu_rc_release_fn)(struct percpu_rc * *)
source

Functions

Section titled “Functions”

percpu_rc_init

Section titled “percpu_rc_init”
int percpu_rc_init(
    struct percpu_rc *ref,
    percpu_rc_release_fn release,
    enum percpu_rc_flags flags
)
source

percpu_rc_destroy

Section titled “percpu_rc_destroy”
void percpu_rc_destroy(struct percpu_rc *ref)
source

percpu_rc_kill

Section titled “percpu_rc_kill”
void percpu_rc_kill(struct percpu_rc *ref)
source

percpu_rc_reinit

Section titled “percpu_rc_reinit”
void percpu_rc_reinit(struct percpu_rc *ref)
source

percpu_rc_resurrect

Section titled “percpu_rc_resurrect”
void percpu_rc_resurrect(struct percpu_rc *ref)
source

percpu_rc_read

Section titled “percpu_rc_read”
int64_t percpu_rc_read(struct percpu_rc *ref)
source

percpu_rc_is_percpu

Section titled “percpu_rc_is_percpu”
bool percpu_rc_is_percpu(uintptr_t pcpu)
source

percpu_rc_getref

Section titled “percpu_rc_getref”
void percpu_rc_getref(struct percpu_rc *ref)
source

Notes on why TOPOC_NONE is fine here:

percpu_rc is designed with the premise of “getting migrated, dec’ing the wrong counter, and all that other business will cause no problem”.

the eventual kill will deal with all the +‘s and -‘s, meaning that if you end up with

CPU0: rc == 9 CPU1: rc == -9

the eventual cleanup goes “9 - 9 = 0” and handles it perfectly ok

percpu_rc_putref

Section titled “percpu_rc_putref”
void percpu_rc_putref(struct percpu_rc *ref)
source

percpu_rc_try_getref

Section titled “percpu_rc_try_getref”
bool percpu_rc_try_getref(struct percpu_rc *ref)
source

percpu_rc_try_getref_live

Section titled “percpu_rc_try_getref_live”
bool percpu_rc_try_getref_live(struct percpu_rc *ref)
source

percpu_rc_is_zero

Section titled “percpu_rc_is_zero”
bool percpu_rc_is_zero(struct percpu_rc *ref)
source

percpu_rc_is_dying

Section titled “percpu_rc_is_dying”
bool percpu_rc_is_dying(struct percpu_rc *ref)
source

Macros

Section titled “Macros”

PERCPU_COUNT_BIAS

Section titled “PERCPU_COUNT_BIAS”
#define PERCPU_COUNT_BIAS (1LL << 30)
source

PERCPU_RC_DEAD

Section titled “PERCPU_RC_DEAD”
#define PERCPU_RC_DEAD BIT(0)
source

PERCPU_RC_ATOMIC

Section titled “PERCPU_RC_ATOMIC”
#define PERCPU_RC_ATOMIC BIT(1)
source

PERCPU_RC_PTR_MASK

Section titled “PERCPU_RC_PTR_MASK”
#define PERCPU_RC_PTR_MASK (~(PERCPU_RC_DEAD | PERCPU_RC_ATOMIC))
source

PERCPU_RC_PTR

Section titled “PERCPU_RC_PTR”
#define PERCPU_RC_PTR(m)
source
Expansion
#define PERCPU_RC_PTR(m) (atomic_int64_t *) (m & PERCPU_RC_PTR_MASK)