-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
337 lines (248 loc) · 15.4 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-3PX6DWT4BN"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-3PX6DWT4BN');
</script>
<!-- End Google Analytics -->
<title>Yang Wang's blog</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="王阳同学的 blog">
<link rel="stylesheet" href="css/style.css">
<link rel="stylesheet" href="fancybox/jquery.fancybox-1.3.4.css">
<meta name="generator" content="Hexo 6.3.0"></head>
<body>
<div id="container">
<div id="wrap">
<div id="nav-outer">
<nav id="main-nav" class="outer">
<a id="main-nav-toggle" class="nav-icon"></a>
<a class="main-nav-link" href="index.html">Home</a>
<a class="main-nav-link" href="/archives">Archives</a>
<a class="main-nav-link" href="/about">About</a>
<div class="main-nav-space-between"></div>
</nav>
</div>
<div id="header-title">
<h1 id="logo-wrap">
<a href="index.html" id="logo">Yang Wang's blog</a>
</h1>
</div>
<div id="content" class="outer">
<section id="main">
<article id="post-How-to-Run-FireSim-on-a-Local-FPGA" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="2023/04/03/How-to-Run-FireSim-on-a-Local-FPGA/" class="article-date">
<time class="dt-published" datetime="2023-04-02T19:57:24.000Z" itemprop="datePublished">2023-04-02</time>
</a>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="2023/04/03/How-to-Run-FireSim-on-a-Local-FPGA/">How to Run FireSim on a Local FPGA?</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<p>Note: This is a translation of my Chinese post, powered by ChatGPT. The original post can be found <a target="_blank" rel="noopener" href="https://zhuanlan.zhihu.com/p/594507816">here</a>.</p>
<h2 id="tl;dr">TL;DR</h2>
<p>Refer to <a target="_blank" rel="noopener" href="https://groups.google.com/g/firesim/c/MGc1qAIYgC8">the discussion and branches</a> here to run FireSim locally on FPGA.</p>
<h2 id="preface">Preface</h2>
<p>FireSim, as a cycle-accurate hardware simulation platform that has emerged in recent years, carries open-source hardware designs and a large number of top conference papers. However, FireSim’s default development platform is based on AWS F1 instance, and the price of F1 is unbearable. F1 is actually a virtual machine equipped with a Xilinx vu9p FPGA. So theoretically, as long as you have a similar FPGA on hand, you can run FireSim locally.</p>
<p>FireSim has also started to support the U250 board through Vitis in the project, but I have neither F1 instance nor U250, what should I do? (I can’t afford it…)</p>
<p>The FireSim community recently provided some clues in this thread. The thread mentions an important development work: NTNU has completed the FireSim port on U250 without relying on Vitis.<br>
Links are as follows:</p>
<p>FireSim: <a target="_blank" rel="noopener" href="https://github.com/EECS-NTNU/firesim/tree/u250">https://github.com/EECS-NTNU/firesim/tree/u250</a></p>
<p>Corresponding version of Chipyard: <a target="_blank" rel="noopener" href="https://github.com/EECS-NTNU/chipyard">EECS-NTNU/chipyard</a>: Pre-release starter template for custom Chisel projects</p>
<p>Following the <a target="_blank" rel="noopener" href="https://github.com/EECS-NTNU/chipyard/wiki/u250_firesim">wiki</a>, FireSim can be deployed on the local U250. (Some steps in the wiki may have issues and need to be modified according to the error messages)</p>
<p><strong>So it becomes contradictory, if I don’t have a U250, how do I use this project? If I have a U250, why would I use this project? It’s simple, understand the principles and then port it to our own FPGAs!</strong></p>
<h2 id="architecture-analysis">Architecture Analysis</h2>
<h3 id="1-firesim is an illusion">1. FireSim is an illusion</h3>
<p>FireSim <a target="_blank" rel="noopener" href="https://fires.im/">tutorial</a> provides a lot of “concise” commands, which seem to be able to deploy FireSim to F1 by just typing commands. But we don’t have F1, what should we do?</p>
<p>FireSim actually packages configuration and compilation options in Python code and yaml configuration, and the final parsed command also uses Makefile.</p>
<p>So we can completely bypass the FireSim command itself, and directly check the source code to know how to compile. The <a target="_blank" rel="noopener" href="https://github.com/EECS-NTNU/chipyard/wiki/u250_firesim">wiki</a> introduction also mentions some common commands and parameters.</p>
<h3 id="2-harware architecture analysis">2. Harware Architecture Analysis</h3>
<p>Since it can be ported to the U250 platform, it means that we can port it to our own platform.<br>
We directly analyze the U250 porting process and <a target="_blank" rel="noopener" href="https://github.com/EECS-NTNU/firesim/compare/1.15.1...EECS-NTNU:firesim:1.15.1_u250">simply diff the U250 porting in version 1.15.1</a>. We can find that the modified part is actually very small and can be analyzed one by one.</p>
<h3 id="(1)-a bunch of configuration files, skip;">(1) A bunch of configuration files, skip;</h3>
<h3 id="(2)-python files, skip;">(2) Python files, skip;</h3>
<h3 id="(3)-hardware platform related codes:">(3) Hardware platform related codes:</h3>
<ul>
<li>platforms/u250/cl_firesim/scripts/main.tcl</li>
</ul>
<p>The overall entry point of the FPGA project, during the compilation process, main.tcl will be sourced in Vivado.</p>
<p><img src="https://pic1.zhimg.com/80/v2-aeacef4c78ce2d25036b48f80e82820d_1440w.png?source=d16d100b" alt="entry"></p>
<p>In it, we can see the addition of various project files, adding constraints, creating block designs, and starting synthesis and implementation.</p>
<ul>
<li>platforms/u250/cl_firesim/scripts/create_bd_2021.1.tcl</li>
</ul>
<p>The script most related to hardware, the style seems to come from the auto-generation by Vivado. After running, you can see the entire hardware architecture of FireSim on the FPGA:</p>
<p><img src="https://picx.zhimg.com/80/v2-32a33bc83fb827ea68e9b93e329cc35c_1440w.png?source=d16d100b" alt="architecture"></p>
<ul>
<li>The core part is the FireSim wrapper: all the generated FireSim code is wrapped inside. The FireSim wrapper inputs the AXI DMA signals and control signals (from PCIe XDMA), and outputs a DDR4 AXI externally.</li>
</ul>
<p><img src="https://pic1.zhimg.com/80/v2-e34de61f66b67e03fc651283f3c713d1_1440w.png?source=d16d100b" alt="wrapper"><br>
</p>
<ul>
<li>XDMA: Default configuration</li>
</ul>
<p><img src="https://pic1.zhimg.com/80/v2-6c7fd07881a9eb22a096335e8e83fc96_1440w.png?source=d16d100b" alt="XDMA"></p>
<ul>
<li>DDR4: Default configuration, if interested, you can try to connect more channels.</li>
</ul>
<p><img src="https://pic1.zhimg.com/80/v2-1aa3e59b33217888c638d944bb4ebd52_1440w.png?source=d16d100b" alt="DDR4"><br>
</p>
<ul>
<li>Clock network: PCIe clk outputs to PCIe; DDR4 input clock comes from the board, output clock drives other peripherals; most importantly, it provides a clock to FireSim, which determines the execution frequency of FireSim.</li>
</ul>
<p><img src="https://picx.zhimg.com/80/v2-0f3802dee693d622aac40c70110b057b_1440w.png?source=d16d100bhttpsb" alt="Clock"><br>
</p>
<ul>
<li>AXI-related modules: Because the FireSim wrapper and PCIe are asynchronous, an AXI clock converter is needed here, and the output AXI DDR0 needs to be expanded to the bit width required by DDR4.</li>
</ul>
<p><img src="https://pic1.zhimg.com/80/v2-49d8c4fd723608a8a978a69f12d8bab9_1440w.png?source=d16d100b" alt="AXI"></p>
<p><img src="https://picx.zhimg.com/80/v2-137381faa4bcf7003b02d22ea5ed5e3b_1440w.png?source=d16d100b" alt="AXI2"></p>
<ul>
<li>
<p>platforms/u250/cl_firesim/design/firesim_wrapper.v, firesim_wrapper connects the generated code to the external block design. If you want to support multiple DDR4 channels, you can modify it here.</p>
</li>
<li>
<p>platforms/u250/cl_firesim/scripts/implementation.tcl, platforms/u250/cl_firesim/scripts/synthesis.tcl Vivado-related scripts</p>
</li>
<li>
<p>sim/firesim-lib/src/main/scala/configs/CompilerConfigs.scala Simply added a configuration with a single DDR4 mem channel</p>
</li>
</ul>
<p><img src="https://picx.zhimg.com/80/v2-cfd28a5d4bcdb78e7202edd64efaff28_1440w.png?source=d16d100b" alt="single_channel"><br>
</p>
<h3 id="(4)-host interaction related:">(4) Host interaction related:</h3>
<ul>
<li>sim/midas/src/main/cc/* Basically imitating Vitis’s driver, actually just reading and writing PCIe/AXI through the standard XDMA driver.</li>
</ul>
<h3 id="(5)-compilation related">(5) Compilation related</h3>
<ul>
<li>sim/target-agnostic.mk Add corresponding compilation options</li>
</ul>
<h2 id="run!">RUN!</h2>
<p>After analyzing the above changes, we can:</p>
<ol>
<li>Create your own block design and modify the tcl script;</li>
<li>Consider expanding the number of memory channels, as the above repo modification for simplicity only used one DDR4 channel. The F1 itself provides 4 DDR4 channels. At this time, you need to modify the corresponding config, and modify the wrapper and block design.</li>
<li>Compile the Linux rootfs, please refer to <a target="_blank" rel="noopener" href="https://github.com/firesim/FireMarshal">https://github.com/firesim/FireMarshal</a>. It actually calls buildroot and cross-compiles to the RV platform, and packages according to the RocketChip platform.</li>
<li>Once everything is done, you can program the FPGA and restart the target device.</li>
</ol>
<p>After starting FireSim, we get a satisfactory output:<br>
<br>
<img src="https://pic1.zhimg.com/80/v2-4213363466cf723675612219f5e68c24_1440w.png?source=d16d100b" alt="cpuinfo"></p>
<p><img src="https://picx.zhimg.com/80/v2-95f63eef71e8033aedd44a2db2a720aa_1440w.png?source=d16d100b" alt="top"></p>
<h2 id="some-shortcomings">Some shortcomings</h2>
<p>As mentioned in this <a target="_blank" rel="noopener" href="https://groups.google.com/g/firesim/c/MGc1qAIYgC8">thread</a>, the existing bitstream is directly written to the FPGA, so:</p>
<p>The PCIe device will hang after programming, and the device needs to be restarted.</p>
<p>A new bitstream needs to be reprogrammed for each new simulation (reason uncertain).</p>
<p>The <a target="_blank" rel="noopener" href="https://groups.google.com/g/firesim/c/MGc1qAIYgC8">thread</a> also mentioned possible solutions: Write a new shell using the Partial Reconfiguration feature, but for now, let’s just leave it as it is, as it’s already exhausting enough to get it running.</p>
</div>
<footer class="article-footer">
<a href="https://yangwang92.github.io/2023/04/03/How-to-Run-FireSim-on-a-Local-FPGA/#disqus_thread" class="article-comment-link">Comments</a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="tags/tutorial/" rel="tag">tutorial</a></li></ul>
</footer>
</div>
</article>
<article id="post-hello-world" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="2023/04/02/hello-world/" class="article-date">
<time class="dt-published" datetime="2023-04-02T08:40:11.333Z" itemprop="datePublished">2023-04-02</time>
</a>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="2023/04/02/hello-world/">Hello World</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<p>Welcome to <a target="_blank" rel="noopener" href="https://hexo.io/">Hexo</a>! This is your very first post. Check <a target="_blank" rel="noopener" href="https://hexo.io/docs/">documentation</a> for more info. If you get any problems when using Hexo, you can find the answer in <a target="_blank" rel="noopener" href="https://hexo.io/docs/troubleshooting.html">troubleshooting</a> or you can ask me on <a target="_blank" rel="noopener" href="https://github.com/hexojs/hexo/issues">GitHub</a>.</p>
<h2 id="quick-start">Quick Start</h2>
<h3 id="create-a new post">Create a new post</h3>
<div class="highlight"><pre class="code"><code>$ hexo new <span class="hljs-string">"My New Post"</span>
</code></pre></div>
<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/writing.html">Writing</a></p>
<h3 id="run-server">Run server</h3>
<div class="highlight"><pre class="code"><code>$ hexo server
</code></pre></div>
<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/server.html">Server</a></p>
<h3 id="generate-static files">Generate static files</h3>
<div class="highlight"><pre class="code"><code>$ hexo generate
</code></pre></div>
<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/generating.html">Generating</a></p>
<h3 id="deploy-to remote sites">Deploy to remote sites</h3>
<div class="highlight"><pre class="code"><code>$ hexo deploy
</code></pre></div>
<p>More info: <a target="_blank" rel="noopener" href="https://hexo.io/docs/one-command-deployment.html">Deployment</a></p>
</div>
<footer class="article-footer">
<a href="https://yangwang92.github.io/2023/04/02/hello-world/#disqus_thread" class="article-comment-link">Comments</a>
</footer>
</div>
</article>
</section>
</div>
<footer id="footer">
<div class="outer">
<div id="footer-info" class="inner">
© 2023 Yang Wang<br>
Powered by <a href="https://hexo.io/" target="_blank">Hexo</a><br>
</div>
</div>
</footer>
</div>
<nav id="mobile-nav">
<a href="index.html" class="mobile-nav-link">Home</a>
<a href="/archives" class="mobile-nav-link">Archives</a>
<a href="/about" class="mobile-nav-link">About</a>
</nav>
<script>
var disqus_shortname = 'yangwang92';
(function(){
var dsq = document.createElement('script');
dsq.async = true;
dsq.src = 'https://' + disqus_shortname + '.disqus.com/count.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script>
<script src="js/clipboard.min.js"></script>
<script src="js/jquery-1.4.3.min.js"></script>
<script src="fancybox/jquery.fancybox-1.3.4.pack.js"></script>
<script src="js/script.js"></script>
<script>
MathJax = {
options: {
enableMenu: false
},
tex: {
inlineMath: [['$', '$'], ['\\(', '\\)']],
displayMath: [['$$', '$$'], ['\\[', '\\]']],
}
};
</script>
<!-- <script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]},
CommonHTML: {
linebreaks: false
}
});
</script> -->
<script type="text/javascript" id="MathJax-script" async
src="mathjax/tex-chtml.js">
</script>
<!-- <script type="text/javascript"
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/MathJax.js?config=TeX-AMS_CHTML">
</script> -->
</div>
</body>
</html>