Skip to content

Commit

Permalink
Auto merge of #59953 - eddyb:soa-metadata, r=<try>
Browse files Browse the repository at this point in the history
 rustc_metadata: replace Entry table with one table for each of its fields (AoS -> SoA).

*Based on top of #59887*

In #59789 (comment) I noticed that for many cross-crate queries (e.g. `predicates_of(def_id)`), we were deserializing the `rustc_metadata::schema::Entry` for `def_id` *only* to read one field (i.e. `predicates`).

But there are several such queries, and `Entry` is not particularly small (in terms of number of fields, the encoding itself is quite compact), so there is a large (and unnecessary) constant factor.

This PR replaces the (random-access) array¹ of `Entry` structures ("AoS"), with many separate arrays¹, one for each field that used to be in `Entry` ("SoA"), resulting in the ability to read individual fields separately, with negligible time overhead (in thoery), and some size overhead (as these arrays are not sparse).

In a way, the new approach is closer to incremental on-disk caches, which store each query's cached results separately, but it would take significantly more work to unify the two.

For stage1 `libcore`'s metadata blob, the size overhead is `8.44%`, and I have another commit (not initially included in this PR because I want to do perf runs with both) that brings it down to `5.88%`.

¹(in the source, these arrays are called "tables", but perhaps they could use a better name)
  • Loading branch information
bors committed Apr 14, 2019
2 parents 60076bb + a173ddc commit 0403760
Show file tree
Hide file tree
Showing 11 changed files with 1,299 additions and 1,732 deletions.
6 changes: 3 additions & 3 deletions src/librustc_metadata/creader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ impl<'a> CrateLoader<'a> {

fn verify_no_symbol_conflicts(&self,
span: Span,
root: &CrateRoot) {
root: &CrateRoot<'_>) {
// Check for (potential) conflicts with the local crate
if self.local_crate_name == root.name &&
self.sess.local_crate_disambiguator() == root.disambiguator {
Expand Down Expand Up @@ -467,7 +467,7 @@ impl<'a> CrateLoader<'a> {
// Go through the crate metadata and load any crates that it references
fn resolve_crate_deps(&mut self,
root: &Option<CratePaths>,
crate_root: &CrateRoot,
crate_root: &CrateRoot<'_>,
metadata: &MetadataBlob,
krate: CrateNum,
span: Span,
Expand Down Expand Up @@ -573,7 +573,7 @@ impl<'a> CrateLoader<'a> {
/// implemented as dynamic libraries, but we have a possible future where
/// custom derive (and other macro-1.1 style features) are implemented via
/// executables and custom IPC.
fn load_derive_macros(&mut self, root: &CrateRoot, dylib: Option<PathBuf>, span: Span)
fn load_derive_macros(&mut self, root: &CrateRoot<'_>, dylib: Option<PathBuf>, span: Span)
-> Vec<(ast::Name, Lrc<SyntaxExtension>)> {
use std::{env, mem};
use crate::dynamic_lib::DynamicLibrary;
Expand Down
8 changes: 6 additions & 2 deletions src/librustc_metadata/cstore.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,11 @@ pub struct CrateMetadata {
/// Used for decoding interpret::AllocIds in a cached & thread-safe manner.
pub alloc_decoding_state: AllocDecodingState,

pub root: schema::CrateRoot,
// NOTE(eddyb) we pass `'static` to a `'tcx` parameter because this
// lifetime is only used behind `Lazy`, and therefore acts like an
// universal (`for<'tcx>`), that is paired up with whichever `TyCtxt`
// is being used to decode those values.
pub root: schema::CrateRoot<'static>,

/// For each public item in this crate, we encode a key. When the
/// crate is loaded, we read all the keys and put them in this
Expand All @@ -73,7 +77,7 @@ pub struct CrateMetadata {
/// compilation support.
pub def_path_table: Lrc<DefPathTable>,

pub trait_impls: FxHashMap<(u32, DefIndex), schema::LazySeq<DefIndex>>,
pub trait_impls: FxHashMap<(u32, DefIndex), schema::Lazy<[DefIndex]>>,

pub dep_kind: Lock<DepKind>,
pub source: CrateSource,
Expand Down
7 changes: 1 addition & 6 deletions src/librustc_metadata/cstore_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -244,12 +244,7 @@ provide! { <'tcx> tcx, def_id, other, cdata,

used_crate_source => { Lrc::new(cdata.source.clone()) }

exported_symbols => {
let cnum = cdata.cnum;
assert!(cnum != LOCAL_CRATE);

Arc::new(cdata.exported_symbols(tcx))
}
exported_symbols => { Arc::new(cdata.exported_symbols(tcx)) }
}

pub fn provide<'tcx>(providers: &mut Providers<'tcx>) {
Expand Down
Loading

0 comments on commit 0403760

Please sign in to comment.