Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth protocol, eth backend, blockpool and p2p integration #155 #175 #230

Closed
wants to merge 96 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
d55d6a5
fix protocol error message memoization
zelig Jan 2, 2015
52706be
ProtocolError -> self.protoError
zelig Jan 2, 2015
98f98f9
add status msg error tests, improve test setup
zelig Jan 2, 2015
7a09ad7
logger rename
zelig Jan 2, 2015
9b203fa
major rewrite and simplification using minimal locking. add many new …
zelig Jan 2, 2015
254fc1f
changes to core/types/block
zelig Jan 2, 2015
acc8ff3
typo
zelig Jan 2, 2015
c610c53
if port is empty string, no listening
zelig Jan 2, 2015
53aa639
jsre executes js file AFTER ethereum starts (allows scripted add peer…
zelig Jan 2, 2015
b39aaa0
add some logging to server dialout and ignored peer suggestion
zelig Jan 2, 2015
add19a6
for blockpool logging peer id is fmt.Sprintf("%x", peer.Identity().Pu…
zelig Jan 2, 2015
63bbf45
fix getBlockHashesMsg decoder (flat, see peer disconnect msg decoding…
zelig Jan 3, 2015
45c7944
protocol and rlp
zelig Jan 3, 2015
a7ac361
fix TestPeerSwitchBack test
zelig Jan 3, 2015
9ab3530
AddBlockHashes call uses lazy rlp decoding so it cannot be async sinc…
zelig Jan 3, 2015
72432fb
protocol rlp
zelig Jan 3, 2015
621062f
fix rlp for blocksMsg and getBlocksMsg in eth protocol
zelig Jan 3, 2015
8566350
make id string only 8byte long for readable logs
zelig Jan 3, 2015
cf93649
automated integration tests for eth protocol and blockpool
zelig Jan 3, 2015
08dd3f0
fix block rlp decode
zelig Jan 3, 2015
d50f33d
fix block rlp decoding
zelig Jan 3, 2015
50d834d
fix block pointer in AddBlock arg
zelig Jan 3, 2015
6d848d0
added test for getPeerMsg/peerMsg - FAILS
zelig Jan 3, 2015
8d77976
move PeerList from protocol to peer, add debug logs (temporary)
zelig Jan 3, 2015
4249bfb
added test for getPeerMsg/peerMsg - FAILS
zelig Jan 3, 2015
9380644
fix getPeerMsg/peerMsg RLP encode/decode, logs. tests pass
zelig Jan 3, 2015
ba0ede1
add shh command line option to switch on/off whisper
zelig Jan 3, 2015
037866d
eth, p2p: fix EncodeMsg
fjl Jan 3, 2015
e771873
p2p: encode peerAddr.IP as RLP string
fjl Jan 3, 2015
3ce2a44
p2p: remove debugging printf calls
fjl Jan 3, 2015
2ef85b4
eth: fix message decoding for working EncodeMsg
fjl Jan 3, 2015
1531915
Merge pull request #1 from fjl/fix-encode-msg
zelig Jan 3, 2015
cde2044
fix protocol error message memoization
zelig Jan 2, 2015
a49243d
ProtocolError -> self.protoError
zelig Jan 2, 2015
5399fce
add status msg error tests, improve test setup
zelig Jan 2, 2015
16f64fd
logger rename
zelig Jan 2, 2015
febd1d7
major rewrite and simplification using minimal locking. add many new …
zelig Jan 2, 2015
aae5cbc
changes to core/types/block
zelig Jan 2, 2015
4757aa0
typo
zelig Jan 2, 2015
d82df7e
if port is empty string, no listening
zelig Jan 2, 2015
4708124
jsre executes js file AFTER ethereum starts (allows scripted add peer…
zelig Jan 2, 2015
f129798
add some logging to server dialout and ignored peer suggestion
zelig Jan 2, 2015
81fe468
for blockpool logging peer id is fmt.Sprintf("%x", peer.Identity().Pu…
zelig Jan 2, 2015
340eac7
fix getBlockHashesMsg decoder (flat, see peer disconnect msg decoding…
zelig Jan 3, 2015
e2b8e05
protocol and rlp
zelig Jan 3, 2015
b768d14
fix TestPeerSwitchBack test
zelig Jan 3, 2015
60d73fa
AddBlockHashes call uses lazy rlp decoding so it cannot be async sinc…
zelig Jan 3, 2015
eb88474
protocol rlp
zelig Jan 3, 2015
0e5e7eb
fix rlp for blocksMsg and getBlocksMsg in eth protocol
zelig Jan 3, 2015
b356a9f
make id string only 8byte long for readable logs
zelig Jan 3, 2015
f91a39e
automated integration tests for eth protocol and blockpool
zelig Jan 3, 2015
74f4456
fix block rlp decode
zelig Jan 3, 2015
a77468b
fix block rlp decoding
zelig Jan 3, 2015
b63ca7a
fix block pointer in AddBlock arg
zelig Jan 3, 2015
5f1bce0
added test for getPeerMsg/peerMsg - FAILS
zelig Jan 3, 2015
50f8ae8
move PeerList from protocol to peer, add debug logs (temporary)
zelig Jan 3, 2015
43877af
added test for getPeerMsg/peerMsg - FAILS
zelig Jan 3, 2015
c218451
fix getPeerMsg/peerMsg RLP encode/decode, logs. tests pass
zelig Jan 3, 2015
b44289e
add shh command line option to switch on/off whisper
zelig Jan 3, 2015
ca719a2
switch off whisper for tests
zelig Jan 3, 2015
50c0375
eth, p2p: fix EncodeMsg
fjl Jan 3, 2015
1869e5f
p2p: encode peerAddr.IP as RLP string
fjl Jan 3, 2015
a7c9534
p2p: remove debugging printf calls
fjl Jan 3, 2015
362189f
eth: fix message decoding for working EncodeMsg
fjl Jan 3, 2015
2916e3a
give moer time to network test 01
zelig Jan 4, 2015
e54bb4e
eth logger imported consistently as ethlogger
zelig Jan 4, 2015
5d957e6
merge
zelig Jan 4, 2015
8dc0888
ethlogger -> logger
zelig Jan 4, 2015
bc59160
fix p2p tests broken due to clientId.pubkey now []byte
zelig Jan 4, 2015
51d7ab7
peer / section interaction redone
zelig Jan 4, 2015
68335e8
p2p: print message code alongside decoding errors
fjl Jan 4, 2015
0bd94fc
Merge pull request #2 from fjl/debug-decode
zelig Jan 4, 2015
c40eebc
rlp: display even more type context in decode errors
fjl Jan 4, 2015
28ba7dd
Merge pull request #3 from fjl/rlp-error-context
zelig Jan 4, 2015
251f37e
restore loopback check for self address for peersMsg
zelig Jan 5, 2015
38edc42
integration tests
zelig Jan 5, 2015
a4c9a99
add dial bool flag to prevent test nodes from dialing out (maxpeers n…
zelig Jan 5, 2015
5720bc6
complete sections on activate chain loop
zelig Jan 5, 2015
1278173
set lastMissing count to depth, so that a reinitialised section conne…
zelig Jan 5, 2015
89763b3
optimize section reactivation
zelig Jan 5, 2015
05f122f
add common js to test with sleep func
zelig Jan 5, 2015
2391eef
add result blockchain to test 01
zelig Jan 5, 2015
4f1f021
add peer switch back integration test 02
zelig Jan 5, 2015
04754ab
add debug logging to peers in integration test
zelig Jan 5, 2015
eb7db45
add 12k chain test (chain not checked in)
zelig Jan 5, 2015
de20891
fix new block broadcast - was wrong subscription
zelig Jan 5, 2015
e39273e
remove debug logging
zelig Jan 5, 2015
81f8372
add TD encoding for mined block for newBlockMsg
zelig Jan 5, 2015
aec5c38
fix iterator passed when singleton blockHash added when new block arr…
zelig Jan 5, 2015
8477815
added tests for several simultaneous mining nodes while downloading
zelig Jan 5, 2015
3e0cf87
clean up debug logger in protocol
zelig Jan 5, 2015
40f54da
add README to integration testing
zelig Jan 5, 2015
3eeb3cd
fix run called without args
zelig Jan 5, 2015
fbb72c8
cleanup debug logs, put most debug as debugdetail
zelig Jan 5, 2015
871dc14
remove debug log line
zelig Jan 5, 2015
a05f02b
fix transaction encoding
zelig Jan 5, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion cmd/ethereum/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ var (
DumpNumber int
VmType int
ImportChain string
SHH bool
Dial bool
)

// flags specific to cli client
Expand Down Expand Up @@ -94,6 +96,8 @@ func Init() {
flag.BoolVar(&StartWebSockets, "ws", false, "start websocket server")
flag.BoolVar(&NonInteractive, "y", false, "non-interactive mode (say yes to confirmations)")
flag.BoolVar(&UseSeed, "seed", true, "seed peers")
flag.BoolVar(&SHH, "shh", true, "whisper protocol (on)")
flag.BoolVar(&Dial, "dial", true, "dial out connections (on)")
flag.BoolVar(&GenAddr, "genaddr", false, "create a new priv/pub key")
flag.StringVar(&SecretFile, "import", "", "imports the file given (hex or mnemonic formats)")
flag.StringVar(&ExportDir, "export", "", "exports the session keyring to files in the directory given")
Expand All @@ -105,7 +109,7 @@ func Init() {
flag.BoolVar(&DiffTool, "difftool", false, "creates output for diff'ing. Sets LogLevel=0")
flag.StringVar(&DiffType, "diff", "all", "sets the level of diff output [vm, all]. Has no effect if difftool=false")
flag.BoolVar(&ShowGenesis, "genesis", false, "Dump the genesis block")
flag.StringVar(&ImportChain, "chain", "", "Imports fiven chain")
flag.StringVar(&ImportChain, "chain", "", "Imports given chain")

flag.BoolVar(&Dump, "dump", false, "output the ethereum state in JSON format. Sub args [number, hash]")
flag.StringVar(&DumpHash, "hash", "", "specify arg in hex")
Expand Down
16 changes: 9 additions & 7 deletions cmd/ethereum/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,14 @@ func main() {
NATType: PMPGateway,
PMPGateway: PMPGateway,
KeyRing: KeyRing,
Shh: SHH,
Dial: Dial,
})

if err != nil {
clilogger.Fatalln(err)
}

utils.KeyTasks(ethereum.KeyManager(), KeyRing, GenAddr, SecretFile, ExportDir, NonInteractive)

if Dump {
Expand Down Expand Up @@ -112,13 +116,6 @@ func main() {
return
}

// better reworked as cases
if StartJsConsole {
InitJsConsole(ethereum)
} else if len(InputFile) > 0 {
ExecJsFile(ethereum, InputFile)
}

if StartRpc {
utils.StartRpc(ethereum, RpcPort)
}
Expand All @@ -129,6 +126,11 @@ func main() {

utils.StartEthereum(ethereum, UseSeed)

if StartJsConsole {
InitJsConsole(ethereum)
} else if len(InputFile) > 0 {
ExecJsFile(ethereum, InputFile)
}
// this blocks the thread
ethereum.WaitForShutdown()
}
77 changes: 35 additions & 42 deletions core/types/block.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,13 @@ func (self *Header) HashNoNonce() []byte {
}

type Block struct {
header *Header
uncles []*Header
transactions Transactions
Td *big.Int
// Preset Hash for mock
HeaderHash []byte
ParentHeaderHash []byte
header *Header
uncles []*Header
transactions Transactions
Td *big.Int

receipts Receipts
Reward *big.Int
Expand Down Expand Up @@ -99,41 +102,19 @@ func NewBlockWithHeader(header *Header) *Block {
}

func (self *Block) DecodeRLP(s *rlp.Stream) error {
if _, err := s.List(); err != nil {
return err
}

var header Header
if err := s.Decode(&header); err != nil {
return err
}

var transactions []*Transaction
if err := s.Decode(&transactions); err != nil {
return err
var extblock struct {
Header *Header
Txs []*Transaction
Uncles []*Header
TD *big.Int // optional
}

var uncleHeaders []*Header
if err := s.Decode(&uncleHeaders); err != nil {
return err
}

var tdBytes []byte
if err := s.Decode(&tdBytes); err != nil {
// If this block comes from the network that's fine. If loaded from disk it should be there
// Blocks don't store their Td when propagated over the network
} else {
self.Td = ethutil.BigD(tdBytes)
}

if err := s.ListEnd(); err != nil {
if err := s.Decode(&extblock); err != nil {
return err
}

self.header = &header
self.uncles = uncleHeaders
self.transactions = transactions

self.header = extblock.Header
self.uncles = extblock.Uncles
self.transactions = extblock.Txs
self.Td = extblock.TD
return nil
}

Expand Down Expand Up @@ -189,23 +170,35 @@ func (self *Block) RlpDataForStorage() interface{} {
// Header accessors (add as you need them)
func (self *Block) Number() *big.Int { return self.header.Number }
func (self *Block) NumberU64() uint64 { return self.header.Number.Uint64() }
func (self *Block) ParentHash() []byte { return self.header.ParentHash }
func (self *Block) Bloom() []byte { return self.header.Bloom }
func (self *Block) Coinbase() []byte { return self.header.Coinbase }
func (self *Block) Time() int64 { return int64(self.header.Time) }
func (self *Block) GasLimit() *big.Int { return self.header.GasLimit }
func (self *Block) GasUsed() *big.Int { return self.header.GasUsed }
func (self *Block) Hash() []byte { return self.header.Hash() }
func (self *Block) Trie() *ptrie.Trie { return ptrie.New(self.header.Root, ethutil.Config.Db) }
func (self *Block) SetRoot(root []byte) { self.header.Root = root }
func (self *Block) State() *state.StateDB { return state.New(self.Trie()) }
func (self *Block) Size() ethutil.StorageSize { return ethutil.StorageSize(len(ethutil.Encode(self))) }
func (self *Block) SetRoot(root []byte) { self.header.Root = root }

// Implement block.Pow
// Implement pow.Block
func (self *Block) Difficulty() *big.Int { return self.header.Difficulty }
func (self *Block) N() []byte { return self.header.Nonce }
func (self *Block) HashNoNonce() []byte {
return crypto.Sha3(ethutil.Encode(self.header.rlpData(false)))
func (self *Block) HashNoNonce() []byte { return self.header.HashNoNonce() }

func (self *Block) Hash() []byte {
if self.HeaderHash != nil {
return self.HeaderHash
} else {
return self.header.Hash()
}
}

func (self *Block) ParentHash() []byte {
if self.ParentHeaderHash != nil {
return self.ParentHeaderHash
} else {
return self.header.ParentHash
}
}

func (self *Block) String() string {
Expand Down
48 changes: 32 additions & 16 deletions eth/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ type Config struct {
NATType string
PMPGateway string

Shh bool
Dial bool

KeyManager *crypto.KeyManager
}

Expand Down Expand Up @@ -124,30 +127,35 @@ func New(config *Config) (*Ethereum, error) {
eth.txPool = core.NewTxPool(eth.EventMux())
eth.blockManager = core.NewBlockManager(eth.txPool, eth.chainManager, eth.EventMux())
eth.chainManager.SetProcessor(eth.blockManager)
eth.whisper = whisper.New()

hasBlock := eth.chainManager.HasBlock
insertChain := eth.chainManager.InsertChain
eth.blockPool = NewBlockPool(hasBlock, insertChain, ezp.Verify)

// Start services
eth.txPool.Start()

ethProto := EthProtocol(eth.txPool, eth.chainManager, eth.blockPool)
protocols := []p2p.Protocol{ethProto, eth.whisper.Protocol()}
protocols := []p2p.Protocol{ethProto}

if config.Shh {
eth.whisper = whisper.New()
protocols = append(protocols, eth.whisper.Protocol())
}

nat, err := p2p.ParseNAT(config.NATType, config.PMPGateway)
if err != nil {
return nil, err
}

eth.net = &p2p.Server{
Identity: clientId,
MaxPeers: config.MaxPeers,
Protocols: protocols,
ListenAddr: ":" + config.Port,
Blacklist: eth.blacklist,
NAT: nat,
Identity: clientId,
MaxPeers: config.MaxPeers,
Protocols: protocols,
Blacklist: eth.blacklist,
NAT: nat,
NoDial: !config.Dial,
}

if len(config.Port) > 0 {
eth.net.ListenAddr = ":" + config.Port
}

return eth, nil
Expand Down Expand Up @@ -219,8 +227,14 @@ func (s *Ethereum) Start(seed bool) error {
if err != nil {
return err
}

// Start services
s.txPool.Start()
s.blockPool.Start()
s.whisper.Start()

if s.whisper != nil {
s.whisper.Start()
}

// broadcast transactions
s.txSub = s.eventMux.Subscribe(core.TxPreEvent{})
Expand Down Expand Up @@ -268,7 +282,9 @@ func (s *Ethereum) Stop() {
s.txPool.Stop()
s.eventMux.Stop()
s.blockPool.Stop()
s.whisper.Stop()
if s.whisper != nil {
s.whisper.Stop()
}

logger.Infoln("Server stopped")
close(s.shutdownChan)
Expand All @@ -285,16 +301,16 @@ func (self *Ethereum) txBroadcastLoop() {
// automatically stops if unsubscribe
for obj := range self.txSub.Chan() {
event := obj.(core.TxPreEvent)
self.net.Broadcast("eth", TxMsg, []interface{}{event.Tx.RlpData()})
self.net.Broadcast("eth", TxMsg, event.Tx.RlpData())
}
}

func (self *Ethereum) blockBroadcastLoop() {
// automatically stops if unsubscribe
for obj := range self.txSub.Chan() {
for obj := range self.blockSub.Chan() {
switch ev := obj.(type) {
case core.NewMinedBlockEvent:
self.net.Broadcast("eth", NewBlockMsg, ev.Block.RlpData())
self.net.Broadcast("eth", NewBlockMsg, ev.Block.RlpData(), ev.Block.Td)
}
}
}
Expand Down
Loading