Implement From<&[T]> and others for Arc/Rc (RFC 1845) #42565

murarth · 2017-06-09T17:29:54Z

Implements From<{&[T], &str, String, Box<T> where T: ?Sized, Vec<T>}> for Arc/Rc
Removes rustc_private-marked methods Rc::__from_array and Rc::__from_str, replacing their use with Rc::from

Tracking issue: #40475

rust-highfive · 2017-06-09T17:30:06Z

(rust_highfive has picked a reviewer for you, use r? to override)

nikomatsakis · 2017-06-09T18:01:54Z

cc @rust-lang/libs -- not sure who is best suited to review this, or what policy you all want to use to decide, but this is splitting up crates and moving things around and I don't think I'm the right person for it. =)

Mark-Simulacrum · 2017-06-11T15:55:08Z

Assigning to @sfackler from libs team to decide on whether we want this.

arielb1 · 2017-06-11T16:06:24Z

src/libshared_ptr/arc.rs

+}
+
+impl<T: ?Sized> Arc<T> {
+    fn from_box(v: Box<T>) -> Arc<T> {


why isn't this using allocate_slice? I imagine an

unsafe fn for_raw_ptr(ptr: *const T) -> *mut ArcInner<T> { }

That mirrors the unsized part of ptr

allocate_slice uses the slice length to calculate the size of the allocation. For general unsized T, size is determined differently. That determination is only made in from_box, so it didn't seem to warrant another function.
Also, offset is used to calculate size, but also must be in scope for the call to ptr::copy_nonoverlapping.

I suppose that both T: ?Sized and [T] could use a common allocation function (like for_raw_ptr). Allocating [T] uses a trick (called "dubious" in an existing comment) to get alignment, but the speed advantage over manually calculating it is likely negligible.

@murarth

The align_of trick is a bit ugly, but I think it's better than hardcoding layout calculations (as allocate_for_ptr does) and is probably going to be well-defined.

I think the "official" way of creating "franken-raw-pointers" is by transmuting to std::raw::TraitObject rather than odd ptr::writes etc.

Also allocate_for_ptr & init_rcbox are always used together. I'll rather merge them and create a ready-to-be-used RcBox.

I picked up the "franken-raw-pointer" method from the RFC's example implementation. It's used, I think, because the unsized T may be a trait object or a slice or perhaps some theoretical unsized type that is structured differently from either of those. Still, it can be changed to TraitObject if that reads better (which happens to work for slice, too).

I agree that calculating offset is not great, but the offset_of macro fails to compile with unsized T and I didn't see a way around it.

Ah, it turns out the Franken-pointer method is used because a ?Sized T might, in fact, be sized, which means that transmute to TraitObject will fail. So, the weird ptr::write thing correctly handles a fat pointer or a normal raw pointer.

sfackler · 2017-06-11T19:53:14Z

I'll defer to @alexcrichton on the implementation details - I haven't done enough work with DST representations.

clarfonthey · 2017-06-12T04:03:38Z

So I haven't read 100% of this but I'm a little confused why exactly this is being moved to another crate to depend on libcollections. I didn't have to do anything of the sort to implement From<&[T]> for Box<[T]> and I don't understand why we have to do this for Arc.

At first glance it looks a bit like you're using Vec to do these conversions when there already exists mechanisms for doing it without Vec.

Mind providing a bit more explanation on that @murarth ?

(in general, it makes sense to not make liballoc depend on libcollections unless they're just combined together entirely. although I do think that Arc and Rc should be re-exported in libcollections though)

clarfonthey · 2017-06-12T04:24:45Z

So I looked through the code again and I'm definitely not convinced that this needs to depend on libcollections at all. I think that separating alloc and shared_ptr is a bit too much complexity.

murarth · 2017-06-12T04:25:40Z

@clarcharr: The RFC specifies implementing From<Vec<T>> for Arc/Rc<[T]>. From implementations on &[T], &str, and Box<T>, indeed, do not require a dependency on collections. If the decision is made not to move Arc/Rc to their own separate crate, I can revert the change and drop the From<Vec<T>> implementation. The others do not depend on Vec or collections in any way.

clarfonthey · 2017-06-12T06:00:08Z

@murarth ah, I see. I don't really think that this is a good enough reason to warrant changing crates at the moment; in the future we should be able to put the implementation in libcollections directly.

Right now all of the Vec <-> Box conversions are done using inherent methods and I think that something similar should be done for Rc and Arc.

bors · 2017-06-12T06:21:37Z

☔ The latest upstream changes (presumably #42419) made this pull request unmergeable. Please resolve the merge conflicts.

alexcrichton · 2017-06-13T17:51:49Z

Thanks for the PR @murarth! My main point of hesitation here is the addition of a crate rather than the removal of a crate. I feel similarly to others in #40475 that the best solution to this is to just merge the alloc and collections crates. Historically it was believed that alloc and collections would remain separate because collections don't necessarily need a global allocator, but I think the ship has long since sailed here.

Would you be up for deleting the collections crate and moving all of its content to liballoc? I would also be ok doing that outside this PR if you'd prefer to not tackle it just here. That way we can (a) merge the crates and then (b) land this PR. Alternatively we could land all these impls here except From<Vec<T>> for Rc<T> for now, and then add that when the crates are merged.

What do you think?

murarth · 2017-06-13T20:34:29Z

@alexcrichton: Merging collections and alloc does appear to be the most agreeable solution. I think it would be best done in a separate PR and I'll gladly get started on that right away.

alexcrichton · 2017-06-13T21:04:20Z

Ok! So to clarify, @murarth you're thinking we should basically put this PR "on the backburner" until the merge goes through? I'd still be ok laying some groundwork here (e.g. with From<&[T]> for Rc<T>) if you'd like! That way some parallel work could happen instead of it all being serialized.

murarth · 2017-06-13T21:47:03Z

@alexcrichton: Yes, I think it will be less work in the end just to wait for the libcollections merge to land and rebase this on top of it. I'm not in any hurry to get things merged as quickly as possible.

alexcrichton · 2017-06-13T21:50:11Z

Ok sounds good to me! Feel free to r? me on a PR-for-a-merge and I'll try to review quickly as it'd be susceptible to bitrot.

bors · 2017-06-14T11:18:37Z

☔ The latest upstream changes (presumably #42433) made this pull request unmergeable. Please resolve the merge conflicts.

murarth · 2017-06-14T18:38:15Z

I did some tinkering and found a basis for an unsized-capable offset_of to replace manual layout calculation: https://is.gd/zHAJTm

The "elaborate" version gives the same result, but avoids overflowing the pointer if the value happens to be at the very end of the address space. I'm not sure if that's a realistic condition that's worth checking for.

Merge crate `collections` into `alloc` This is a necessary step in order to merge #42565

murarth · 2017-06-15T18:57:47Z

Rebased. Also significantly simplified allocate_for_ptr using the above described method.

alexcrichton

Looking great to me, thanks @murarth!

alexcrichton · 2017-06-15T19:49:33Z

src/liballoc/arc.rs

+            let v_ptr = &v[..] as *const [T];
+            let ptr = Self::allocate_for_ptr(v_ptr);
+
+            ptr::copy_nonoverlapping(


Should this use Arc::from_box(v.into_boxed_slice()) perhaps?

Or alternatively, could this call copy_from_slice (or share the implementation there) before it's followed by set_len(0)?

alexcrichton · 2017-06-15T19:50:25Z

src/liballoc/arc.rs

+}
+
+impl<T: Clone> Arc<[T]> {
+    fn clone_from_slice(v: &[T]) -> Arc<[T]> {


This and copy_from_slice are only used in one location, right? Could they just be inlined into From below?

alexcrichton · 2017-06-15T19:50:58Z

src/liballoc/arc.rs

+#[unstable(feature = "shared_from_slice", issue = "40475")]
+impl<'a, T: Clone> From<&'a [T]> for Arc<[T]> {
+    #[inline]
+    default fn from(v: &[T]) -> Arc<[T]> {


We currently try to avoid exposing the usage of specialization right now, mind changing this to use a local ArcFromSlice trait which can be locally specialized?

alexcrichton · 2017-06-15T19:51:22Z

src/liballoc/arc.rs

@@ -1211,8 +1360,49 @@ impl<T> From<T> for Arc<T> {
    }
 }

+#[unstable(feature = "shared_from_slice", issue = "40475")]


Right now we unfortunately don't have stability checking on impl blocks, so you can go ahead and tag these impls as #[stable]

alexcrichton · 2017-06-15T19:52:01Z

src/liballoc/arc.rs

+}
+
+#[unstable(feature = "shared_from_slice", issue = "40475")]
+impl<'a> From<&'a str> for Arc<str> {


While we're at it, should we perhaps have From<String> for Arc<str> as well?

alexcrichton · 2017-06-15T19:52:38Z

src/liballoc/arc.rs

+impl<'a> From<&'a str> for Arc<str> {
+    #[inline]
+    fn from(v: &str) -> Arc<str> {
+        unsafe { from_arc_utf8_unchecked(Arc::copy_from_slice(v.as_bytes())) }


I'd personally be ok just using transmute here, exposing from_arc_utf8_unchecked is unlikely to be stabilized in the long run I think.

alexcrichton · 2017-06-16T14:50:27Z

Looks fantastic! If it's ok with you I'd like to double-check with @rust-lang/libs though that we're ok on merging this. I'm going to tag this as S-waiting-on-team and this'll come up during our typical triage which will occur next Tuesday.

aturon · 2017-08-22T23:29:30Z

@bors: r+

bors · 2017-08-22T23:29:31Z

📌 Commit 8e0d01b has been approved by aturon

bors · 2017-08-23T01:59:58Z

⌛ Testing commit 8e0d01b with merge 944675da03018dc4746039b8076dc169fe26ae22...

bors · 2017-08-23T03:24:09Z

💔 Test failed - status-travis

kennytm · 2017-08-23T07:10:37Z

@bors retry #38618

bors · 2017-08-23T11:19:08Z

⌛ Testing commit 8e0d01b with merge 6decb846e857cfb9f99f2a3c74834be18d69cf0b...

bors · 2017-08-23T13:46:10Z

💔 Test failed - status-travis

kennytm · 2017-08-23T13:52:47Z

@bors retry

A Mac is unscheduled, an another Mac failed to download the LLVM archive correctly.

[00:00:38] Attempting with retry: sh -c rm -f d9e7d2696e41983b6b5a0b4fac604b4e548a84d3.tar.gz &&             curl -sSL -O https://github.com/rust-lang/llvm/archive/d9e7d2696e41983b6b5a0b4fac604b4e548a84d3.tar.gz
[00:00:45] tar: Unrecognized archive format
[00:00:45] tar: Error exit delayed from previous errors.

BTW Travis is having problem with Macs currently:

To address the infrastructure instability and reduced capacity issues on OSX builds, we need to do an emergency maintenance bringing all running OSX builds down. The jobs will be restarted as soon as there is a slot free.

bors · 2017-08-23T20:42:34Z

⌛ Testing commit 8e0d01b with merge 2aabed68e72aa9c0e64f6b5cfd40aea152ee4cb7...

bors · 2017-08-23T22:32:09Z

💔 Test failed - status-travis

alexcrichton · 2017-08-23T22:36:33Z

@bors: retry

spurious failure of a test that failed to start Spurious failure in test_process_output_fail_to_start #43283

bors · 2017-08-23T23:20:38Z

⌛ Testing commit 8e0d01b with merge 560b6ca...

Implement From<&[T]> and others for Arc/Rc (RFC 1845) * Implements `From<`{`&[T]`, `&str`, `String`, `Box<T> where T: ?Sized`, `Vec<T>`}`>` for `Arc`/`Rc` * Removes `rustc_private`-marked methods `Rc::__from_array` and `Rc::__from_str`, replacing their use with `Rc::from` Tracking issue: #40475

bors · 2017-08-24T02:32:23Z

☀️ Test successful - status-appveyor, status-travis
Approved by: aturon
Pushing 560b6ca to master...

Changelog: Version 1.21.0 (2017-10-12) ========================== Language -------- - [You can now use static references for literals.][43838] Example: ```rust fn main() { let x: &'static u32 = &0; } ``` - [Relaxed path syntax. Optional `::` before `<` is now allowed in all contexts.][43540] Example: ```rust my_macro!(Vec<i32>::new); // Always worked my_macro!(Vec::<i32>::new); // Now works ``` Compiler -------- - [Upgraded jemalloc to 4.5.0][43911] - [Enabled unwinding panics on Redox][43917] - [Now runs LLVM in parallel during translation phase.][43506] This should reduce peak memory usage. Libraries --------- - [Generate builtin impls for `Clone` for all arrays and tuples that are `T: Clone`][43690] - [`Stdin`, `Stdout`, and `Stderr` now implement `AsRawFd`.][43459] - [`Rc` and `Arc` now implement `From<&[T]> where T: Clone`, `From<str>`, `From<String>`, `From<Box<T>> where T: ?Sized`, and `From<Vec<T>>`.][42565] Stabilized APIs --------------- [`std::mem::discriminant`] Cargo ----- - [You can now call `cargo install` with multiple package names][cargo/4216] - [Cargo commands inside a virtual workspace will now implicitly pass `--all`][cargo/4335] - [Added a `[patch]` section to `Cargo.toml` to handle prepublication dependencies][cargo/4123] [RFC 1969] - [`include` & `exclude` fields in `Cargo.toml` now accept gitignore like patterns][cargo/4270] - [Added the `--all-targets` option][cargo/4400] - [Using required dependencies as a feature is now deprecated and emits a warning][cargo/4364] Misc ---- - [Cargo docs are moving][43916] to [doc.rust-lang.org/cargo](https://doc.rust-lang.org/cargo) - [The rustdoc book is now available][43863] at [doc.rust-lang.org/rustdoc](https://doc.rust-lang.org/rustdoc) - [Added a preview of RLS has been made available through rustup][44204] Install with `rustup component add rls-preview` - [`std::os` documentation for Unix, Linux, and Windows now appears on doc.rust-lang.org][43348] Previously only showed `std::os::unix`. Compatibility Notes ------------------- - [Changes in method matching against higher-ranked types][43880] This may cause breakage in subtyping corner cases. [A more in-depth explanation is available.][info/43880] - [rustc's JSON error output's byte position start at top of file.][42973] Was previously relative to the rustc's internal `CodeMap` struct which required the unstable library `libsyntax` to correctly use. - [`unused_results` lint no longer ignores booleans][43728] [42565]: rust-lang/rust#42565 [42973]: rust-lang/rust#42973 [43348]: rust-lang/rust#43348 [43459]: rust-lang/rust#43459 [43506]: rust-lang/rust#43506 [43540]: rust-lang/rust#43540 [43690]: rust-lang/rust#43690 [43728]: rust-lang/rust#43728 [43838]: rust-lang/rust#43838 [43863]: rust-lang/rust#43863 [43880]: rust-lang/rust#43880 [43911]: rust-lang/rust#43911 [43916]: rust-lang/rust#43916 [43917]: rust-lang/rust#43917 [44204]: rust-lang/rust#44204 [cargo/4123]: rust-lang/cargo#4123 [cargo/4216]: rust-lang/cargo#4216 [cargo/4270]: rust-lang/cargo#4270 [cargo/4335]: rust-lang/cargo#4335 [cargo/4364]: rust-lang/cargo#4364 [cargo/4400]: rust-lang/cargo#4400 [RFC 1969]: rust-lang/rfcs#1969 [info/43880]: rust-lang/rust#44224 (comment) [`std::mem::discriminant`]: https://doc.rust-lang.org/std/mem/fn.discriminant.html

RalfJung · 2018-10-08T13:28:59Z

Turns out this has a soundness bug :( #54908

libcollections was recently merged into liballoc: rust-lang/rust#42648 I went ahead and also added an "alloc" feature which no_std users can use to opt into liballoc features (i.e. any code using Vec). This should have no effect on anything but no_std usage. It does make it possible for people without allocators to use curve25519-dalek if they want though. Might be nice for "bare metal" development. All that said, from what I can gather liballoc, while not "stable", should likely stick around for the forseeable future. Some backstory on the liballoc/libcollections merge here: rust-lang/rust#42565

rust-highfive assigned nikomatsakis Jun 9, 2017

Mark-Simulacrum assigned sfackler and unassigned nikomatsakis Jun 11, 2017

Mark-Simulacrum added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 11, 2017

arielb1 reviewed Jun 11, 2017

View reviewed changes

sfackler assigned alexcrichton and unassigned sfackler Jun 11, 2017

murarth mentioned this pull request Jun 13, 2017

Merge crate collections into alloc #42648

Merged

bors added a commit that referenced this pull request Jun 15, 2017

Auto merge of #42648 - murarth:merge-alloc-collections, r=alexcrichton

258ae6d

Merge crate `collections` into `alloc` This is a necessary step in order to merge #42565

alexcrichton reviewed Jun 15, 2017

View reviewed changes

alexcrichton added S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 16, 2017

arielb1 removed the S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). label Aug 22, 2017

bors merged commit 8e0d01b into rust-lang:master Aug 24, 2017

murarth deleted the rc-from-slice branch August 24, 2017 04:18

murarth mentioned this pull request Aug 24, 2017

Tracking issue for From<&[T]> for Rc & co #40475

Closed

est31 mentioned this pull request Sep 9, 2017

im beta regression #44446

Closed

KodrAus mentioned this pull request Sep 12, 2017

Add Shared type to fix inference fail on nightly bodil/im-rs#2

Merged

parkovski mentioned this pull request Oct 3, 2017

No shared conversions for other string-like types (Path, OsStr, CStr) #45008

Closed

carols10cents mentioned this pull request Dec 31, 2017

[NO MERGE] Comments from nostarch on ch 19 rust-lang/book#1087

Closed

tailhook mentioned this pull request Jan 10, 2018

An attempt to convert into std::sync::Arc<str> from std::convert::From<&str> tailhook/abstract-ns#10

Closed

RalfJung mentioned this pull request Oct 8, 2018

From<&[T]> for Rc creates underaligned reference #54908

Closed

Implement From<&[T]> and others for Arc/Rc (RFC 1845) #42565

Implement From<&[T]> and others for Arc/Rc (RFC 1845) #42565

Conversation

murarth commented Jun 9, 2017 • edited Loading

rust-highfive commented Jun 9, 2017

nikomatsakis commented Jun 9, 2017

Mark-Simulacrum commented Jun 11, 2017

arielb1 Jun 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arielb1 Jun 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfackler commented Jun 11, 2017

clarfonthey commented Jun 12, 2017 • edited Loading

clarfonthey commented Jun 12, 2017

murarth commented Jun 12, 2017

clarfonthey commented Jun 12, 2017

bors commented Jun 12, 2017

alexcrichton commented Jun 13, 2017

murarth commented Jun 13, 2017

alexcrichton commented Jun 13, 2017

murarth commented Jun 13, 2017

alexcrichton commented Jun 13, 2017

bors commented Jun 14, 2017

murarth commented Jun 14, 2017

murarth commented Jun 15, 2017

alexcrichton left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcrichton commented Jun 16, 2017

aturon commented Aug 22, 2017

bors commented Aug 22, 2017

bors commented Aug 23, 2017

bors commented Aug 23, 2017

kennytm commented Aug 23, 2017

bors commented Aug 23, 2017

bors commented Aug 23, 2017

kennytm commented Aug 23, 2017

bors commented Aug 23, 2017

bors commented Aug 23, 2017

alexcrichton commented Aug 23, 2017

bors commented Aug 23, 2017

bors commented Aug 24, 2017

RalfJung commented Oct 8, 2018

murarth commented Jun 9, 2017 •

edited

Loading

arielb1 Jun 11, 2017 •

edited

Loading

arielb1 Jun 11, 2017 •

edited

Loading

clarfonthey commented Jun 12, 2017 •

edited

Loading