Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate x86_64-fortanix-unknown-sgx-lvi run-make test to rmake #129055

Merged
merged 1 commit into from
Aug 23, 2024

Conversation

Oneirical
Copy link
Contributor

Part of #121876 and the associated Google Summer of Code project.

The final Makefile! Every Makefile test is now claimed.

This is difficult to test due to the uncommon architecture it is specific to. I don't think it is in the CI (I didn't find it in jobs.yml, but if there is a way to test it, please do.

Locally, on Linux, it compiles and panics at the llvm_filecheck part (if I replace the x86_64-fortanix-unknown-sgx with x86_64-unknown-linux-gnu, of course), which is expected.

For this reason, the Makefile and associated script have been kept, but with a leading underscore.

@rustbot
Copy link
Collaborator

rustbot commented Aug 13, 2024

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 13, 2024
@rustbot
Copy link
Collaborator

rustbot commented Aug 13, 2024

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

@jieyouxu
Copy link
Member

jieyouxu commented Aug 14, 2024

I'm going to be real with you, I have absolutely no clue about sgx. So I'm going to request help from our sgx target maintainers after I take a look and try to guess what the test is trying to test.

@workingjubilee
Copy link
Member

This is about verifying that LLVM cross-compiles SGX code with load value injection defenses.

It should be possible to make it work after building the cross-compile target.

@workingjubilee
Copy link
Member

This target is tier 2 so we should be building the cross-compilation artifacts every time anyways.

@Oneirical Oneirical force-pushed the fortanix-fortification branch from 1f9cc33 to dc650f6 Compare August 15, 2024 14:19
@workingjubilee
Copy link
Member

So: per policy, tests don't have to pass on SGX, a tier 2 target, in order to merge this. As this is an SGX-only test, we can just merge this through.

I do know, however, that the SGX maintainers do in fact run the CI regularly and add ignore-sgx, etc. and are reasonably responsive, usually.

@jethrogb @raoulstrackx @mzohreva Can we get your review?

@raoulstrackx
Copy link
Contributor

Thanks for tagging @workingjubilee ! I have a day off today, but I'll try to look at it on Monday.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the port, I have several questions (that we will need help from the sgx maintainers).

@jieyouxu
Copy link
Member

jieyouxu commented Aug 16, 2024

Thanks for tagging @workingjubilee ! I have a day off today, but I'll try to look at it on Monday.

@raoulstrackx Thank you very much for helping out! ❤️

@Oneirical Oneirical force-pushed the fortanix-fortification branch from dc650f6 to aa15961 Compare August 16, 2024 15:03
@rustbot
Copy link
Collaborator

rustbot commented Aug 16, 2024

The run-make-support library was changed

cc @jieyouxu

src/tools/run-make-support/src/env.rs Outdated Show resolved Hide resolved
src/tools/run-make-support/src/env.rs Outdated Show resolved Hide resolved
@Oneirical Oneirical force-pushed the fortanix-fortification branch from aa15961 to df2ec7d Compare August 16, 2024 15:26
Copy link
Contributor

@raoulstrackx raoulstrackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment this PR breaks CI. See comments. Can you please check it? You shouldn't have to have an SGX machine as the enclaves don't need to be executed.

@Oneirical Oneirical force-pushed the fortanix-fortification branch 3 times, most recently from 9e0ff38 to ffdde64 Compare August 19, 2024 15:33
@Oneirical Oneirical force-pushed the fortanix-fortification branch from ffdde64 to b5a6a38 Compare August 20, 2024 15:44
@raoulstrackx
Copy link
Contributor

There are other issues as well. Please apply:

diff --git a/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs b/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs                                                                                                                                                                                              
index 84c292d019d..506b9424d39 100644                                                                                                                                                                                                                                                                                       
--- a/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs                                                                                                                                                                                                                                                               
+++ b/tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs                                                                                                                                                                                                                                                               
@@ -41,13 +41,13 @@ fn main() {                                                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                                                                                            
     check("std::io::stdio::_print::[[:alnum:]]+", "print.with_frame_pointers.checks");                                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                                                                            
-    check("st_plus_one_global_asm", "rust_plus_one_global_asm.checks");                                                                                                                                                                                                                                                    
+    check("rust_plus_one_global_asm", "rust_plus_one_global_asm.checks");                                                                                                                                                                                                                                                  
                                                                                                                                                                                                                                                                                                                            
-    check("_plus_one_c", "cc_plus_one_c.checks");                                                                                                                                                                                                                                                                          
-    check("_plus_one_c_asm", "cc_plus_one_c_asm.checks");                                                                                                                                                                                                                                                                  
-    check("_plus_one_cxx", "cc_plus_one_cxx.checks");                                                                                                                                                                                                                                                                      
-    check("_plus_one_cxx_asm", "cc_plus_one_cxx_asm.checks");                                                                                                                                                                                                                                                              
-    check("_plus_one_asm", "cc_plus_one_asm.checks");                                                                                                                                                                                                                                                                      
+    check("cc_plus_one_c", "cc_plus_one_c.checks");                                                                                                                                                                                                                                                                        
+    check("cc_plus_one_c_asm", "cc_plus_one_c_asm.checks");                                                                                                                                                                                                                                                                
+    check("cc_plus_one_cxx", "cc_plus_one_cxx.checks");                                                                                                                                                                                                                                                                    
+    check("cc_plus_one_cxx_asm", "cc_plus_one_cxx_asm.checks");                                                                                                                                                                                                                                                            
+    check("cc_plus_one_asm", "cc_plus_one_asm.checks");                                                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                                                            
     check("cmake_plus_one_c", "cmake_plus_one_c.checks");                                                                                                                                                                                                                                                                  
     check("cmake_plus_one_c_asm", "cmake_plus_one_c_asm.checks");                                                                                                                                                                                                                                                          
@@ -66,9 +66,11 @@ fn check(func_re: &str, mut checks: &str) {                                                                                                                                                                                                                                                              
         .stdout_utf8();                                                                                                                                                                                                                                                                                                    
     let re = regex::Regex::new(&format!("[[:blank:]]+{func_re}")).unwrap();                                                                                                                                                                                                                                                
     let func = re.find_iter(&dump).map(|m| m.as_str().trim()).collect::<Vec<&str>>().join(",");                                                                                                                                                                                                                            
+    assert!(!func.is_empty());                                                                                                                                                                                                                                                                                             
+                                                                                                                                                                                                                                                                                                                           
     let dump = llvm_objdump()                                                                                                                                                                                                                                                                                              
         .input("enclave/target/x86_64-fortanix-unknown-sgx/debug/enclave")                                                                                                                                                                                                                                                 
-        .arg(&format!("--disassemble-symbols={func}"))                                                                                                                                                                                                                                                                     
+        .args(&["--demangle", &format!("--disassemble-symbols={func}")])                                                                                                                                                                                                                                                   
         .run()                                                                                                                                                                                                                                                                                                             
         .stdout_utf8();                                                                                                                                                                                                                                                                                                    
     let dump = dump.as_bytes();

@Oneirical Oneirical force-pushed the fortanix-fortification branch from b5a6a38 to 6a9e9fc Compare August 21, 2024 14:27
@Oneirical
Copy link
Contributor Author

Oneirical commented Aug 21, 2024

There are other issues as well. Please apply:

Oh dear, I should have watched where my editor's multiple cursors were going. Thank you so much once again! Changes applied.

Copy link
Contributor

@raoulstrackx raoulstrackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the tests all pass. :)

tests/run-make/x86_64-fortanix-unknown-sgx-lvi/_Makefile Outdated Show resolved Hide resolved
tests/run-make/x86_64-fortanix-unknown-sgx-lvi/_script.sh Outdated Show resolved Hide resolved
@Oneirical Oneirical force-pushed the fortanix-fortification branch from 6a9e9fc to e276d22 Compare August 22, 2024 14:22
@Oneirical
Copy link
Contributor Author

Oneirical commented Aug 22, 2024

Now the tests all pass. :)

I deleted both Makefile and script.sh as they are no longer needed, let me know if you'd like to keep them.

Thank you so much for all your help!! This would have been much harder without a fortanix-rust maintainer.

@jieyouxu
Copy link
Member

Now the tests all pass. :)

@raoulstrackx thank you so much for helping to review and test 💚

@jieyouxu
Copy link
Member

I deleted both Makefile and script.sh as they are no longer needed, let me know if you'd like to keep them.

They're still in the change history, and now the rmake.rs version reimplements them, so we shouldn't keep them.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this LGTM as well. Feel free to r=me after PR CI is green.

tests/run-make/x86_64-fortanix-unknown-sgx-lvi/rmake.rs Outdated Show resolved Hide resolved
@jieyouxu
Copy link
Member

@bors delegate+

@bors
Copy link
Contributor

bors commented Aug 22, 2024

✌️ @Oneirical, you can now approve this pull request!

If @jieyouxu told you to "r=me" after making some further change, please make that change, then do @bors r=@jieyouxu

@Oneirical
Copy link
Contributor Author

@bors r=@jieyouxu

@bors
Copy link
Contributor

bors commented Aug 22, 2024

📌 Commit e276d22 has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 22, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 23, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#127623 (fix: fs::remove_dir_all: treat internal ENOENT as success)
 - rust-lang#128876 (Ship MinGW-w64 runtime DLLs along with `rust-lld.exe` for `-pc-windows-gnu` targets)
 - rust-lang#129055 (Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake)
 - rust-lang#129386 (Use a LocalDefId in ResolvedArg.)
 - rust-lang#129400 (Update `compiler_builtins` to `0.1.120`)
 - rust-lang#129414 (Fix extern crates not being hidden with `doc(hidden)`)
 - rust-lang#129417 (Don't trigger refinement lint if predicates reference errors)
 - rust-lang#129433 (Fix a missing import in a doc in run-make-support)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 23, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#127623 (fix: fs::remove_dir_all: treat internal ENOENT as success)
 - rust-lang#128876 (Ship MinGW-w64 runtime DLLs along with `rust-lld.exe` for `-pc-windows-gnu` targets)
 - rust-lang#129055 (Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake)
 - rust-lang#129386 (Use a LocalDefId in ResolvedArg.)
 - rust-lang#129400 (Update `compiler_builtins` to `0.1.120`)
 - rust-lang#129414 (Fix extern crates not being hidden with `doc(hidden)`)
 - rust-lang#129417 (Don't trigger refinement lint if predicates reference errors)
 - rust-lang#129433 (Fix a missing import in a doc in run-make-support)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit fe87433 into rust-lang:master Aug 23, 2024
6 checks passed
@rustbot rustbot added this to the 1.82.0 milestone Aug 23, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Aug 23, 2024
Rollup merge of rust-lang#129055 - Oneirical:fortanix-fortification, r=jieyouxu

Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

The final Makefile! Every Makefile test is now claimed.

This is difficult to test due to the uncommon architecture it is specific to. I don't think it is in the CI (I didn't find it in `jobs.yml`, but if there is a way to test it, please do.

Locally, on Linux, it compiles and panics at the `llvm_filecheck` part (if I replace the `x86_64-fortanix-unknown-sgx` with `x86_64-unknown-linux-gnu`, of course), which is expected.

For this reason, the Makefile and associated script have been kept, but with a leading underscore.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

6 participants