From dde65ed73996cfa8af5f426670a876ed70273fda Mon Sep 17 00:00:00 2001 From: Anuradha Weeraman Date: Sun, 13 Jan 2019 23:11:25 -0500 Subject: [PATCH] Report and updated weights --- Report.md | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++++ bananas.py | 2 +- weights.pth | Bin 10624 -> 10624 bytes 3 files changed, 64 insertions(+), 1 deletion(-) create mode 100644 Report.md diff --git a/Report.md b/Report.md new file mode 100644 index 0000000..01c8529 --- /dev/null +++ b/Report.md @@ -0,0 +1,63 @@ +# Project: Navigation + +This is a project that uses Deep Q-Networks to train an agent to capture yellow bananas and avoid +blue bananas through deep reinforcement learning in a Unity ML-Agents environment. + +## Learning algorithm + +Q-Learning is an approach which generates a Q-table that is used by an agent to determine best action +for a given state. This technique becomes difficult and inefficient in environments that have a large +state space. Deep Q-Networks on the other hand makes use of a neural network to approximate Q-values +for each action based on the input state. + +However, there are drawbacks in Deep Q-Learning. A common issue is that the reinforcement learning tends +to be unstable or divergent when a non-linear function approximator such as neural networks are used to +represent Q. This instability comes from the correlations present in the sequence of observations, the fact +that small updates to Q may significantly change the policy and the data distribution, and the correlations +between Q and the target values. [1] + +To overcome this, experience replay is a technique that was used in this solution that uses the biologically +inspired approach of replaying a random sample of prior actions to remove correlations in the observation +sequence and smooth changes in the data distribution. + +## Model architecture and hyperparameters + +Fully connected layer 1: Input 37 (state space), Output 32, RELU +Fully connected layer 2: Input 32, Output 32, RELU +Fully connected layer 3: Input 32, Output 4 (action space) + +The hyperparameters for tweaking and optimizing the learning algorithm were: + +max_t (750): maximum number of timesteps per episode +eps_start (1.0): starting value of epsilon, for epsilon-greedy action selection +eps_end (0.01): minimum value of epsilon +eps_decay (0.9): multiplacative factor (per episode) for decreasing epsilon + +## Plot of rewards + +Below is a training run of the above model architecture and hyperparameters: + +``` +Number of agents: 1 +Number of actions: 4 +Episode 100 Average Score: 3.97 +Episode 200 Average Score: 9.51 +Episode 287 Average Score: 13.12 +Environment solved in 187 episodes! Average Score: 13.12 +``` + +The plot of rewards for this run is as follows: + +![Plot of rewards](https://raw.githubusercontent.com/aweeraman/deep-q-networks-navigation/master/graph.png) + +## Ideas for future work + +Further optimization of this architecture can be performed by training with different hyperparameters to +get faster and better learning outcomes. A couple of further approaches to try out are: + +* Double Q-Learning +* Delayed Q-Learning + +## Reference + +1 - https://en.wikipedia.org/wiki/Q-learning#Deep_Q-learning diff --git a/bananas.py b/bananas.py index 5f684ee..c15d574 100644 --- a/bananas.py +++ b/bananas.py @@ -9,7 +9,7 @@ import matplotlib.pyplot as plt # https://github.com/udacity/deep-reinforcement-learning/blob/master/dqn/solution/Deep_Q_Network_Solution.ipynb -def dqn(n_episodes=1000, max_t=750, eps_start=1.0, eps_end=0.01, eps_decay=0.995): +def dqn(n_episodes=1000, max_t=750, eps_start=1.0, eps_end=0.01, eps_decay=0.9): """Deep Q-Learning. Params diff --git a/weights.pth b/weights.pth index af70317a330fa9ae9b74e77ba828a6536919901e..4693f74a287641e6f7af1346598d5611d30992a4 100644 GIT binary patch literal 10624 zcmZ{Ic{tVI*EX3#QAr|2Nr}ji@qE^jLPaDh8j&GHIiWeBP?F3;2_-3ID3v&$y(A4v zX--laG-%MM(!{Ip@B6&h^FG(}e*QSub?&qGz3#Qwea=~HNA*jP3;vyJ|3fqAu#2ie zR6nQJ5`C|mgaf67D}~~H3p@t)eG(iP9v9^81 z(0iq@-*ykVJ}`WJsGnz8;0C`ip~PmPq>hJF9~c&}YIV3!%CVn^MBlr>mEl5ZM@bLK zzZ|PpE*Hu;N=8KV#q{Y56YS>`-WMfIZuV+lztw)ewt+t3aze2Qu0nC0e**aX7#Xbf z3tS!$F6`$MzG}5kfPrU3cwlgtoabu4Wf6hFzMkQJE5lZ;_FQ8uCzR09@sRnepl_x3 zazCNuf40ck2e0xDcl!@0CzSF~>|0|r+0evrvZ<-5xvAk~6Jw$DKWcqKBZM+e1(MF; z!v1p{#T+#qwc~`cj@pjmaYDHreW<*vaDYeuzWsldEerGx6AldM3mYIA@IPpcEX>V~ zErbdIGLD*k`xO6Sixm!X6ps}S{>wDPRjBlDj>i9$+#=A^M8*}{C}YTIgEb)9W^!m*9j8;FZypyCI4UaUo-v>Fa3Kg(tZY-e-rF` z9`3P+I}2p#)`%DEo}MfDj93RFrzt^niW?NW4ZycP&fMWdbqY|CK<`dFSeMd(-fOGi z=l09&$r5I_)(;cMB_N+~ic_O}@Vo95 z{H(p2TBG8aqup0fm@)%1Kg}oU3){)7yo?E zrS?sjaZ1F0yx@w3@=@46KUq}2)tZ&NJcpEcW3>HT$le@!%&UKtr15GV8H zO0dOr8U|`L!0(OCa4W-{HygTCWNp>Veb_k#3>6S=>ThCiMx27&&}Y!BU<~ayF0q(% zYbmIvkU4aBuwS{0(Wbf-Y+het4?n);K7P$-TH_9bqH7bAzFNvwN!(^)fB#z#Z%&q+bb}tKk6qPcFg8;B8R0?k&jgNQErk2>%T4g61B@+BD6Z zAoN*-)5n*x_s_R;hc?FZigI_MdtV;hzOh3fqx2l^FO{P}lUN!fF$*`#8X`>UPbHZz z*~BSX`1$@W?n0Lj)wS%}>R#?}?+W&P&vR&$s%H`- zbTQ+g8Lx8Mi~LS#P*O=3S68wQPL_$Iwfa!3x6*{q*AO3;55ehqiMTL4l|0H)VXt>S zU6`XsBFAFbR@BAE0xIe3hi8|MB1-%U z0|v}NmnIRjy(2}vo~0<#N<+iqdu-5gYuNW~2gTJH5Wgb{^rxO;0YCS^;M*1aI;9mL zJLonS3%U6C;bxRE8;Kf6{2+U_Gre;ALq{%dWFeZTXok5D9&WW{>l_mxH|;E4zscdd zgN=M(W+=ZkeE`k)E``ZY_JF;=ItI(X;GJfK!mf%1bpG`uZp__4d~L84E8ni+jPL7$ zl)O4kEI!AZSlUwXof1^C*ux4R3&_pL8b+;Li305zwD9yiup8(PF@yZ5@y1-Zc`cVT z4WqEb&y+?9qT%b^92^zbpWd}5vw0&Eq2_ubXVj7gV=Kp@bhsX7o&E^tKP<#d}#zXza>-*#GJ zWC_W;%xK)PTo!&-87ppebCcuNL8iJPcq}@Ep1XHazV-sNd+9}YNB4^KZjYq_NxtaY z(*r$SFZkaFBiXII;i8S7pNS6H-hqAh({O%&6`U126Ur8i$8+CCafQ7%Az9jp&8o4% z;GiKaOMg98l<7d#HnG}jziqhU*BSn8cNKdUe@2iMJ&xB5jpgDpcJQOle__VEkI;p< z7jW8aE`EtpWA(r7Xrc2pP%0DA#`;pW)iMn>Sg2CUb7Oo~H;sn!GwI6m6D;DvTY<$% zXUsg3D|lFSf#r9VvRuDW^tHYW&N@_}eI4U_msPWK(qU|ZLIV^#0;5@60FTuys{#ekdh|OP)@X1H# z;CZcRQFN&mi`SdM4tI^AfI)Sz=*)1|6to7eOQcbPS}SXBmxIOCIW&K93SL`cO(TUP z@MZUI$eUpWqp!Z_HXX1--^Ua2ajGY|g&d*V%N59Kb`hIJ;yCUj!N3=~sJ~SSMT)NQ z!BL-nywXB`;z!tRaDbaO--DdHmC?>T29{n}24lR-A=a}PUKYqw!rf2oW1BR#_+)ZP z8G~uT8zo$lH-j!Y$YZ?OMA|>x2hN@jXE(N8q}6xFQ})TSpP?Nz-jRYehWK6NuM!vuQ!y&IH87ukeAPSoC13f6y;__*+MFh1-V zGii@wQl~$|{+Us%E%FOp>aB_Q6N1|K@Nk+$9rwBNiPeblDXtiHFKnBAawc;|4D-*6z4X`k_Ck29ii z=-L&idppwV<-mPdpsOJ&uU5tTV^^Zj%K7|>Yb!v-^%^8?tzgk=xzr_hfPM!ifLOB) zj!fDOP9r?|r^80!1d2U$MCd2Se`l z@f6ge$$5o-1dSW2aC)N`J`_u#lq_dh5T?Re&;N&?u5^8OOFKLE!xjqo(WLx;3np)(ModJWL z=hMf|SmaW+;pBc+*i$Hhr{4BJc}F~{UrYsmF$0p8k)({MQMBK98eRFUPMOw~Y)Z)j znr6OSaIP)@%|q6cXnG@W=TgDjTbHr)(P!Z2y?tjsyvc(Tvs2mJcb3jWbr%dP<(CxZuxLe5L}nZzPO!Zb-xM1%PjdXuf!<8B!e$>Jp(UJYf^rX zF%1cI=UT!%p>Fp*uwOfz{c@;gD#qbtk-dvq=ugFsgBK7le*{hi>XTUaRM@1OMmbVN zFr`Eev+r)F6l-Jjj>|y%<}52$%>%5c>?j?ZCCgmXzp{GmVpx_o2(Mh4hR;ZiwR@do z+=fBiH!&$F{FVa~ll5Vz9pVR5bt+k<&nD6&?*1MR!t=h87)B6P*GJ12tPTQlzSmPT+YUCKI?#RY;P`rQ=?;Zpf$tI?Ka5oKos>6!% zj=`?jQ*2U`JxrO>FERHk&79d_wu--hoJE$RLMUHJ(#H~2Ir6n$YqPt-}c z#S=QDwz4P{GSinHLypr%Q~tpO)OA&*yfjaO8%3D1XErHCnn6^MDh6jYz=(m-#ImxX zt!p3_choY+5ys@bWC7H_YDD7!Bbm{_>!9-H7MMlGu%b^Zc@`1@3P!i+OIHKgy^;aj z8{=_k`v%PKY=iZ`<ur?i@y^kB?`8?TSa^xeoBfuXFer?+|3eZOZtH&?4TLFB5vgbcNVrI1mr)}P)J=H(|CJgbWZaU_@z z+D|xA9wp*?Sm?VK;H#3uY$V50mANmwrrZXtQ{!mfj~Spb+nU$#osO1OvS6=u1#0hY z!nZSqQu68VY_zusqF3!^pZ}a;4uf^@Tk;O7-1r(aBUVF!-V)4I7>s41`7EcihHW&e z#*NWhI32_J#;ReoNpU%h*M81K<(4?m4@Bk7JFwcMgayBP$W))5;B=MCnWI-P7*DZ8 zg$WHbFdf+TiXr4&cZYTMx5BYbRdDTA6C66{#BJ`a!&S4-aTOuJ6?p`qzl|NsN_NC< z|5lds>ZcVuYfYspa`^518@Sby$0hlGVC;PW$iLYGc|DieIGdfQzo3Mg)CZtuXdEow zq6ZJoSh443YD`DgnO&O^OC|Exv5m#R1z|e7+K>hIw_ij0D+%1<7>zm}QLJ&=6*xJA z_!J2nG>k9@mrv`^b+S5!ni`{1stjo=N5Y=N>Li_{$mTrDWL2s5*fM=OYt_^O#bM>B z6K~1H(`!KBIF{u!+rXN3Rd%G}lBmgTIyuA`lC*3W-(GA(RY`N)6Ze~w{Ko*Hv&|Jcx5R}-9cMFJnceMD7iQ`r0C zeh^k8Pd@83u=TPXO(GMJ7}5rFi}KjVaZ@1R?lYDeKb8uf24bi%kMb>gS;)r^%%&?TX;_*15NF1L8UV;`YF&61q)z=&EwaXEzlP*5m$H1N(Qwr5C@dE8u=hv=_`KXpF|jIm!h1Pf zd{}CwuXq3+M8)Ipm~pf*>oN7a{gWx%2>GYyT__@KBTf`tVUII?=*Fi@Aiv=z8`Y81b1oiCH*a`S^(86( zL{10yx!#HWxK+cqnAhU6IpfIT>;PITFyKG6T%+IZO1Ns#Be*5Ik7=zlB8S!lydBUC zy`wU~C~_vVe*BBS7sc4X%X>wpg^RgmGN0f++YW0tslta0P`CM(mcmYj2Y z6-%tKr5z=5RJ!jXq?yIDsUgi!dP}!&9#^1hb5nA8-^Q|@Sdm54K`x_W9V?vekETK^ zc0#!ntTNJRVA^1^{3*+7g^{r8XA?U=uN;l^ezG3{TQRu)Fq@*AO=fB7bZvhIn=?Q_ z<*u4|<{^-egBZjzak3h3#`p63*@FjTalUXgGka7Irb|0{$=4^z_|rTXyKoU3oqn9W zX*xYVe+#B)zUOS7n6XOveQd!BG2E9$T$As7Y}#7KZq8DrhJs#J`L&QbzfEIZd1vXa zmJ2KR{S!`W3_@M61g@?#3W8d;iu9t6Q^y2xTAYyw+46FNs%ciB;P}<@hD{zaVLhAU zQwT-v^Z7FGd~|=B0&3yWB>FX-o=)6Lrq4%`$*w23PVXoio}gaKmxRO3p-R+cFNLEv zbh6Z04_Vv(AlUR&lU13l!G2FAi4|?2+R_U&ROc;x{(b;o_w{#lQ4Q1G za|9QC??)3-l<3K+G&1f@qrh-|Ja4~<&Kio>Zn)D=P#u4l-*&_p3MZ;TsGb#wxo?Ay zE>5BcF*o^9M=kjqr2y>iOIUvD9X$N?PM|+zKkS&>VRcRA1zYAiouCySIqQftL@re`euqN*X0x3-g)++gy zncUsPjeL5SB_3-Nge-D{ZvHAedeIp&hW}={8@ABiA4LK+S8ocgR7U6euW;^BIRxhG zqDMt2`0F}=UwHzv^pc}5*WcE7)Vvfet3Hg$^?OCLE)S`t`?tu_qp)`A-T*ec`wiQp z6b-YD2N1lR&DQ-`!#hvi3&WRn^8V9?(SCyr_A_xKC~~=6{$vinG&gbfCY}6vP=UAW zfAdzuIC$ATo)S;QGl%D5=vr<-LHFEAa#$Xlze1t5K&z5TXWW2G>Lr-fu!9}99z(?& zWoWL1SZ(T;tL*uzW87Qc*8G6q1s85_#in0Am?66X6dZ)uT9?EY zty881&6CW&=@4xp#>7XRgyi0Q(dv<|m}xD8k9RllGb>^Qm(ES+qunLx`hlBl?Vfnl z>`4-IYF4vOe|ehh+eQy>O29(FP%c{g2Mn@R8ad0k4+%ANP+Y3mkB4F-!a}ckN#NCfK991 z;PJdAKs)LfK0XGqcFWL~v~Yjs3obR)mtNg!pmSNS_{3}+J92zHi-X%t{@G{FebyuX znEGsPnfC-dJF|q74Z6Yjdm40g>M{y1c*O>}A+%Yf;v$K-*?psy=oa?wMR?QC=-<>>vXW?tBls zRCd5B_d75+S`k*SxXpZ>JI}1RcAsDPaTryl26Bs|C3)#fUJw^?3g)Ru!>AQMxDTQZ zu($jIa`i9xVGjOK>3SAEZ>WORNA01`>J7h4^DX!C^(Ks*_JXrB7y%wP=2Ey{7oR&Z zn~U90#s`nlV~US{urUX-aCLSAInI|M>HV`vW+9$erHO3^NlJ;O?X`c;RRq=F=hJPqT7#Scs3@VR~~K4*FNZl+e7Yy5Z__r z<~yu8Z#M~+`BRW=N^R~$C(1Z&f+?1Xa5XERw-(dDI~x_Pq-R?SRxXpL(T5obBCP10 zw*q}{=8(jcD1D?ho~-`^JBQw9n{374+QlsLG9H4@Tq?lo!Xb8(Ga|!%D$uUV(*foW zC8i&l^x75F>s`RAiq!Z$Qx-sIgqa{|>~xy!br8BK6UmC&JMCh?;x>4}3_ z?S#6r+D|J_g4Nd=5X~!qd+sH$?8#*=Jt&30_4)ySGWi@FOZfqFpqw{Xo`iL=6EPu8 z2CpROVB`HL-gnksDE1kLM`b$T`?6%P{8SFcWda=3N)D&r4sc*1oUZstaK-4zt+q>I|Hkq%{h!`GufxXP#6kl4=xBln$vU5(q>){=XI ziSHCJJ$@ju2LYt>vYUODb6`J0>e(SlYj8WHPg+&MVB=)Xh1yTyC6+8@!FT%6B7-fk zGrNf||5U);yg34IxD3F&+~e%Y!hG6&$e+d+O4ZuRy3o-*OF44u%Uw-@DS8j(g0il# z*Xw)PPvLfc{(~^qtlh=PyPO$}Y2aks9$?PcFk0x4MeZ+dvz>+$Y9mr(;M%v<@an=w zKI-O2!Rv8qZ1$#MsL?h8o+ubmkCqvyG|PwSds&cI@GZuVNnyK2inHD4mg4SyZgf1` z7+Y=hS*vM`R1iFRj;TVwIyyQ45;+g9fEya)oX39zjF8pKD3pqu@E zwrGDZefhJGpS-fL&o`Cq^XMk9Ff8Qki-%ypnt{ao$&p%O7PMVlL}49^V2#yaYRa?1 zco$K?QITvO z$wSbT%OF3}hU=={!PX|maa!5WVH|S;6^}B$_2*5pkg0V8Pq0!21F zdpGPGlno6#4`O8Lq{hrNK zJmlcdyc6V|{0V%mN5G7jo9tM_L#!y=#o*8uc1BB&ZMgV`y}Q*9Ew)eO7Y9CNdd)A{ zu^|rd=EV!BY)oY*w<7I-pUzZ{ZG~}l8rbsA9Zs*XWf9d${P6m6=60!yjkOZT){TeA zIW>>TzEy+7Ynk|b;|JbcOOofV61!{}3pd*)fK1L(2$2bA$%pp|e!Z;VYTA~f{hUFy z;)i?qL23$Uy;hmU^v&rSmeJs7>Md{!ia;la*|<=)kzJL`C7lUvv@5azp4wh!@6^A- zp(<~@VAgUWxcP`Q-?D5sn-t!LEeexKE#(|5RQLuO=7acN zry(RhNrk+o-r%lIP=FstlxbZ4Xga8&O+nYoq4%yNw67aRN2keCW}ZE9CAmy=C6oX9 zaU}%=2GWO;rTqA(dSusplKZf}nr$CbMK`82aNpgZK&jVM?r22>Nm|^4{KIv8;P7aa z@ED52-~3?>P43|JD}wZ=?%+>KiDAx*;iC0Ymb9&KD$_jnofk^ZMgDE0z{RPDM+4SAqLXZ9^paGY-jD(q(cQs^%!}b+)&%yUu)=cI)1ff>>vE7$j$-mjdX!!Bk}l32hVEur(0M6J zVDYS+^1Ic+xF?ER?3@Bu>@t|`Mtxe|tkO~yB_qzd%!m4R{WFpxKqLZ?%ExK8aC zIQPQR+6!+!)8XXrq&>T}R$aUmhD82^X=YzwYYNHE?`{Uo`udXHvM@X)4t`) zDVKM1CUP3gJ>wpel>izop-xtc-Qcyq12R9hvPB^SsQ=;LFpiJGDH#Ayju8Dt44W-nS%DU@)NX6ip*Nr#dKZoe_d#dRC~SS$2D$qm zgFs~)1egpL*kz?~ddqh*jo&Ky^+f?5Y=WV&lJ3sbvfr>`TU|jsi~Nu72%E zFK6y@WDIXqaf^GsS`W)keh060bLoTDDd@`k!_8Qm&nB-NMbGBj^5V06u&z`Fj}44O z9nXjSghO#PSBGlwm(DA(rnPQtn!P3cnikBQCKd6Gi!*TgAZM~4w~K1;#tZ6&duZsd zcMw@D4!6by!kT_p_(@tN%yCae%^B(CpruvJeSSZU?TZ_T5$!s7psPO`EH4JJC#rDu zR4I!LFv9q}KHmoGLH2eFNS(cwdl(l30{6ib8eYnL<<&9CZU9zKxdA7{EMdytfxJY} zAy8^s%ChX+*zURp7ISDW4mf&+g4V>cK;DP_Tq9{M+Efi!_sQegd+Xs{?sNDd?!$$; z%!1CQnQTLz6FYLRKl7ZFBPu;p05&ay`Ctummcn`&<*cG9_l#jq>p93-_?c@dC02jp z10SoFj#bg8X!C(++R-adBN!J1}nTaF|h4BjT1^`Mmvz9CDRor+*# z!njpVBGN5-LZXWnwb6x*aA&YPX`OtoPI2ymv6J)IvdVb2vSm3go4~{MxS62z zh~ZC>9qsT8gqM$0$TRIAsQyrd!mu@X{c;EYlE zJE5Pz*1TzEQ+s7mr+ybJvimBUxZxu2ekTk{`6#|Hemp+hFU!Z;U1a}WLy1M4OF*&h ODyW;94r|}e!v6yy*-oAS literal 10624 zcmZ{J2{@JA*S>ko5K4&*Wk{K2I?r0CR8(l7fd(oGMV!!xlnBv4X+nfd8A>GQ+52cv zd8yE-Bq|M(W)ao@^uF)!`hVYdecw6Pbm;S8rio%Mwa z#+t^6xxEz^en#;`j69V^L_|V2g@lLrh6MR+_S+N|7_yNkDs14XAp9pNFg!fS&&O}$ z+Q8Ktd148kazZeCTd1E;Sm1WQFrIi6Pr}GkQV50xglr1uNxF!6iVJ@QZVcy1xkz|Q z{NV`M=+BdOk%)*8j_E5LCdkh>TsTUY-26>z{Wkfnof+sGF2@s@yt%(Q>TI&<;w=pbalh0;LIi9$Yk*D+@1;UoA z{rz|n|8$Xa3JO^r?(q*O$CLC_61K6hHlJ!S)!N>6s?}6mdrO|w-)enBBY4to*%I@@ zc`^%KL|k-S^y7K5F8VH_@jSUiAu8|A8{sJ>?V+8~KMx#hs`6?;I`vGe@=mmCMe?(%Qnpk~bcMO=qxNPY|MF@|4P&PSK9bL(zO3D>A3TB z|CMI-A8C632bZO-y}31Sf{-gl*mL3^E(sT*GW~y)8T{chbmtlUtIX;jWt09dHalB8 zYo4)??aw?X|IHT1oAO7Q$sZVVmqzIs3OLbm z7jv}t7Tcm-;b7Iy#XGak!onNPux8hGsF6QGpRMR1O`qk#rD7JE$$8?od|S*&x=eoG zONN~N1q|7B5~O5;YXbthNHf_9J3XV(_|r>t=ZC+0BI6F!0GLY6;a zrzgpLr-06m*)X|@i%|F*Ob@*zE0(B`^OGkL>&AmbJM=2#o6Ui1Y6>L%9Acc`Nc?sA z8g%%@kV7`R>1mmjxL893yo^=pzQx7#Zp2-|nUiB-M|C>XpH6@~$)aF&&w%fL%a&bn z{UK4;3?vOzUXWpbh#cDdfRQ-)0L;I>g{rYC7*cVI?dm4{`|c57wtWn~lT;4lH~KpBGyTP-K;W%Bpjg z9ag5F9#6w*jRUy@GhoRyJpGK5_kW8}7==)1a(L z_{pmWUixm~clT#7ww?v-Xj~67TI*r?U=j+hUBR5zF$||KlB!QJ5?tQeCb$*O5^WmW#c%a7X5uzWF_S-eI=m3) z*N)WMz>3tkF@@XPN$xmbR&vI~OWT}(L8y%h8o8E3SD-kmMP32xXcb&{Fbym<_K-y< zoX9vvoR!FT;<)ae4&l5HOo>bZJHhfHN1xlp%>SVQA7zqBVR`^#{w2S*F>W-o-h2<} z8ePJ{H#g|~6LmE96`z*2-W9yOkj5WTgHFmyADdVXF*By2$&f(ijJAp z2yAN!wq5%MGp}f2s_q_Cou5e?EJNYs{30;^56bzf$C3V9!UwEQ?@v`^;tbS8u{;+vB0? zj1)x+2EUo`zcv1;3kDiKStX@EUTn#ySU@j?ZFa+5T?Qmd7 z6>)hcJ{=Q){#;`Wq6tj=^J4hDE`*us{+mcN>>+i}H<2SxGx=(2pP|yfn6Z1D532$i zLGgnFPS=`-vmeC3fjK@<8R`r3qCRt4oOQ^Kgk>4#zFjP}jjN)+*S)7Ar={vVw(Ta{yEpP{luMbrj*aj&K?JWH zR%2F#+~qHMwhrD-KLs-y+rZ&*9&_e-1Sy{&&W-j`5=cza5@?Ak2>LFI*DW{@3rC|x z(X{3od^+?UN?zBpgZ;_ik{1jp(g!W-g)l_UGZE?eY#-=`ZtsxbniArp0xTcTt~Nj_(g*I&+xFFIa_B#ZHl>5nt%X zH=4wot4goxQke_uB*g;RcUV9Jcy+1*N zyl2sh&aV!U36lj+jy!{mh;K~EJ3gG#*~b~Xy_!LbePH#wns96O!cwX2%WaVm4q+Z65RfLnb@7sSsjzjK|ZTPGP%)1-9thqhatqVrJ36q-Cwd;f+La1w=*v|^I>)*xoWOs4|Ja^_; z_gL)bIl!fJWhD5AIdv`WAotCVke*e8VH?C) zGo!1dY)dMglJb}|Uvz@mo@MMVZ9n26CqYGYEYK@%0Zfw}#d?SJL7wyKIZq<0be z1ixg))c&gRW!8ehmru;f2dCiv+AI9@W6wCQo+da`RTG^(9l@XkQMBhgNj`NAHP7C` z=l2qE&*r_j$=r@In}gBJIsuNYRYS#7ThZawfNx6Q$l}dh4{YOW1OLt?#%5X+$vW_jhK^<`MPTvXd^G#ivI_ z^BB2PW#skb7Dn2}gM+763kUem`UVUw=kk| z8Kg{C0bcTqN$~wX5*;W4-c#)y7R6T(&-LPj?S9BETcZJb0U35=L$I(Zo7Q$MCidel!!yA4*q+yxbXmIq-gOTSd;7nRR6KyEq zJgJT692BI$ma!v9+l1Gw&72*4X{EE|G)E1DkAFB*wGtNl1Tf9E)0s;di;=sg8d^Lg zapph__{Ogy2NvZagt!XshKLG^xfkiSMK=T|O&*fNWe#kAxg_(^^JJ~9^Kn@6PzH~c zOW<(n9%gr04&iGy)27li);}y9lFu13;|BTQm2nGh4kxe?Ei>6cc@H={uLO#QwKy@q za-edm2w8CKCQi(i5~w;-s+aVUw%>jsIDYLo6h#ihv4@{vjiVt3i_Qjf@n@iNwG)zU z_rr*7DzHCDg0xK=X6-+I;ON{~0ZEr$gVXmmkg>Z3+7cyjuw*W5s=UDr{qi8U(=_0m zj5rQD9)=#ZG^l)}0D5DCn82EMe9qIawP#BU?_NSPeWj7c#FQf_gPS7evCxS%I7Z@B0H6}nb}9CJcz`Y zVhcJ_&xDSwd&3Fyze?BM_yAj9?}qf%onYW|3#MoELY7rNlXCPMXP}j314sYn6RSP! z?Ql&JdoF>Kwaf_OLQ0srpTAkLPyLJ&Gn%!GIm&$WSqrJluQJozI+!!^7i&v4$)Z%0 zB)TP^|h?H|DYhT<%>n*W@wz_^g_R%l;c&$p^%}iVJLQ99l{J6<~)4u?EyvyO9 z=XEIPxC2w)mO`uw;F7!=c+&ELKX>0f2hR3Im$J znRN$;$r!0jnoiFA>yPeB(lEZ|G$byc2HrKg z?DsQY7`+RJpj8RsutEw3%PyjxKZ5CmTct$C=eS@(cp9_XA)>PQK|&$%i2vXaa*;oINRS%$MS#?V7oTJSl5s1_z)|WWJQ?!@zJ3$MpWBjAvsRrhU0X_<54zGjp`!(!e=Wo% zI`O0^UX*4iC(}pfbJ1YPidqb`@we1?)2!hUl$Y8M&$nD;W=x4hNy99vRP2Ydm*3`` zNOZ+QMuXlIXS9m!`y)K*75?wCl=I$Sw^aI^$xPq8dN^Rr-K#yD6YqULByQ znnYr)MF`iu&*9snDfB|U46zcP1v^eU;+OMEbVEC32Gz6;SD6F*DJ{iC)f{N^AD+A$OKbQuj&I(75&n zC(3jcRLK-D%QJ2>WeKKuR7RS{IGfXwaZkyAbOcLYWC}MgCGJ)#bob~CqPu?+p6}>{ z7X?iuPneRKvB#P$;knWojfr@=$`c(T25{f#MHt*No6+_uM%P{yoV5Qb^CP{SbFohy zUv&#$()jzN@zXiX(>8--t2ijMZ09>?lwx@KH+Worjy$@SKsLoq#PDW)6!|wVx3w>x zScltiqf3IxVbw)w=QS38PdrY3TzF31__N8}T~2iNle^HV;3+T|7|)s)dEkCUNjgPK zlWaZehdLX6kYkz!M71Z6oVEEuHoX{N5ABe{Uc-<0?iZK5&7DFEXHBG=U2W-%6U(Xh zsAOVYXhz@pi=xJjI->Rb0sHysaVj}+xUd?sYtvxdV|$TA z>x|`Qq>rVUJN^RYx3z4DWesZ)cAFf$e3sF3&O>|c)uijxS#%v;0ikn`f|lwH+CN)T zu+8-|Rc#z6(2MM#p1D}kI5WaC$iOk4K;uhZ2Z_7upHSz5@!|Drc?WI$XHzP zv_%zVgDXg!Mito$g|u!#98R~m0AHR`%8ydPoj0uL9K%nqwKdqQp|1v~IjD^GZ-eJ2IMK(gR(Rs(y_ry;H(m%?c&&$M-UcZw?cmo+{vft0OLP1@J9f zmbJPqff1tim|8XhGf6GU`rQj*>!;Jf_t%Nz>;h1WO5}VyrA@yLm65$SBx#Dpboy9f zDLd)ZJGhqH$t=ESL}NnDXok}Q)@dF><^y{)-|j~apL|V@tF5EWC7zrGVTb8LqD}i# zr=b4*<<#=r2I@Tb6X|F)rTZkN&@Yh#q;=$V3=BL4wU}@qw;V3@m3LyM|j|< z!*_|xOqN}tv zzLenudp#_04M&Tpy>L87gNzEOf! zlUjiY;_#=Y5`0+a$E;n}%UPxUl{H(MMj}I=fN!cS6~wQBG-d}5yFFzSn|!I(oQ+gv zNfv22ufR0uZK30q8NlS-vuKWXF60|z)5j@UFgIERKIV)EdG9H(Y=A>FGiA|j;aM_U zsL!D&g_?|ANH6YRNHzEB(w)Nwl(JuwmIXsQ5Do}h8R7kb(tClOHfYI zK5U&k05|RCgFYT&?%o+9by0iJ__;3H&GEtGTFddFiohOrDDfDtpEV>T_4oRE-!=S`kP5mWm5}%5+F{Qv`kU#+|Iu-9rmS z>&Q;6cw#y^AH>=D)M{uc4rp`H>0|&+I-X8Ton!Fkr9y_E-U8vFQ@H&dF_;n;M>Dj8 zXj9ZJ$Uf9hy+53yWvLBx_gF)YjJh)BepjcGSr^gvj3@F^Jt42`E>Ta(VzYzdF}+0q zp}eE;XMmYiWqx2BULW6$9s;hnLyNP+%-oXJ1%%3P3Pa-0r?Z^xE^(=?P@NVYC(g*##5 z?2~s1Fml&=T)D1-ILy97)ZK&dtmIjY*(w3CUGwSgRd3)$O(iXA_<{}D*P!0yHoX6= z$EelZN4dZcbe7(GFymH|cT67Wuh>QIyexpTyAQC2M-@@7vk0bWh@p?fR1A_aA>udk z$t2(R#BSkB*x@8aFBipPaG^YTee4Cw>JZS)JxV?+&nL6CU8LN@0d$SWI#61mPCACC zLFJF7q-HRgt*_XH2NXvW(S0Y-)2xy4J290Sxs8L7ez8QV?I7sg2qj$MnI$u^j*Kh$ zL|gn*8PDH$$S3VOR2^-KMYdYl8P!O?tBQifyVrETett}zkQ@h9mtzGnncr{qWErpW{9)p40S&}Kf zpCez=LbxYN;fUyBkXZc`whMEivj@Y8Zf6PkyxfT1stU)GM&{JZVhJfAvjsNsvTU6A zRw8;vnI5jVEBuy1c}-E$KCqPhkW2%a3zbaYeSTs99s7E zvE=$lJbB-rX4ZUUG!snd&}AE1S6oSAx!)L%Pn5hVJj)mf>oBp&Rs6MDVL0Nro*?su z4RyP+h@NwCCchubQtK}S9H)!XJkt=&YA>Kg8r{Hs6-YNs)}t@2q}j202SGjl1<{}N zkrhc=PYqtRv3ZqOaQ!-Y!G~fNQvPbAV#_KaawwLHZgZki3#>q)H|a%|u;N0Q|gzlZ9;I9!prflOX=9-Qj)S)J*%^v5GE+TWc;QVL2@I!c17 zG>AgYiy|VIyqs!3s3gDcB_r2M3P(OQLo*Z;#NAe*F&TG7chM)a9xxydOUkcPNa-<_+!R+ zV(#*UEze)Z_H)Xxd4rn6Ig8zbs2Lih<0DHW@9iNXhbrjJ{StWbzz7fqDx7$S9wJvh znYm|_4<`C@g6e`mkX++V=ijt|UmdsE6W?S7?bEN4wM&dhqNyF1v1y~vWJhynOxsKL z3@PC%V<*%d$R!)|_M(jWDBQbX0ZG?u=Q{@Z3saXDiLT!RT+#Z7DBs(I&!(g>evx%p z+#@VQ^hpY2-i*YY?q0U7gz`HVn+fACEox`@0lb2vDgW&orl|cIS>*Tx6wI`^``;xA zW9`jU!4oiW`6lSX4=}M;Si>&qgxsPX#OnQh^8NS@w*PUngImA^yclIf0&3%lSf>e^ z%o~l~i{h~AeI7m=BoO~12um*{f_tb8ZkE&}g}d{CWXwh1t#NpAfiORv|=2*n3Cqwd{2bpE7?ZSOvS^_p1D)@vh#b#WW?Q(R0%?*1l! z?LCRx-@b$=-s!CLu(cr3N*P~f3eTAhu5`eE0X=e>PxZckgn{u9^vUgdlGl2dRVk6i zfyDEyf>{(XUu8{iJ<+7EciF;zg`1QH2}qH ziMOt>KI%S+c(A&#vay^g*1km3z8_(#V*}}t11DJ5%nHWo=qoBEokBAEENDfSGg%rH z!{oVrr0a8ML1;=b+*uX|$ty+?Mdd=WCa{J0+sos}2QtEY?o_CI9M4o;Hxg|7G64QD z4b-)B0IarY3+&(63!3eZLs{BR@UhqC_;&v!&X4bag7g~)fGrnRSt}t z(nBqKrRmP7P#VL`76f`MW;9ge1Q!GE!IAVN`uOBMhRhs^D(gF0URwg4ow0^=$ozz@ zJ5=B-sM8rAw$oJoFnXnb0d46X$G>izPvWZjVfbDl&2-TsYU^C+foGd(TEkX0%p{Kb zZ;vJ$WR}4|d^#v5w-a9X0ouKO0v&Sj!*7ieRC0|szVcNg8IASiltnM>@(CuvjT32f z%Y8C6s+4$I-)9?L4^heXWISqXLtk_^<1ObGu)k0V2R>$y$3MqHfbLXUs3D3OVfysU z4xm2$7tl>b z4@S&?h$FY3hm!e+Am-n^hzApgnT{{<=@JQ&fqUSHpE~Z_u$%sFQpUH|iLlt>C5$op zi|8D91Ck3RxhhucQ6*Mf@K(xFuvl?5#?lk8&2R>GO?1c5;aptNuZf3^4`S%%Ofaa@ zrO$M9a7>K^N}b$=8iuv>S?X~dy`zaLtY;zfEeA`!igBIhC!o=KKRo813dFzxS4mtX z(_;t0b*C`4c^1d~j$B0EaV7Bk`bYSo!v!xcR^tfDH_}5P`|&}<2Y8de85a$f;G-vn zxJ%a)4eay5=(Q8R{u)67#~j8Jw`XC@&gnR;I*De7|HSF_zBKUFQA|1&f#aomd-NV`&-}SlX_9O7n{$?;2jYYjcXS&Mo7QJJ6 z8vE2j&};o0jJ^Mckr_}SD_T-<_?A09Y}D?Z(Ny(`m`J5b`wl z3Vm8&gduY-K!)T@ZthGLQ{4~H+Y%YHC?2pry&4L1#2`001>KD=fX~xXln-#i_4=CJ zjOb)+t5$;PZfaCgJp#YQ2TE>NaXsctC(GDjU(QP;92Q3{D0D}Ka2nX