4b9c3bd39a
swarm/storage: disable open tracing on indices ( #19578 )
6 years ago
9b0d1b9ab2
swarm/metrics: collect metrics on datadir disk usage ( #19576 )
6 years ago
db83ba4067
cmd/swarm: skip export test on windows builds ( #19555 )
6 years ago
c8a77d8604
swarm/metrics: track runtime metrics ( #19557 )
6 years ago
7744241886
swarm/network/stream: add pure retrieval test ( #19552 )
6 years ago
9b1543c282
swarm/network: update syncer metrics
6 years ago
297fa1855c
swarm/network: measure addPeer and deletePeer to know if Kad rearranged
...
swarm/storage: remove traces for put/get/set (#1389 )
* swarm/storage: remove traces for put/get/set
* swarm/storage: remove Has traces
6 years ago
84dfaea246
swarm: instrument setNextBatch
...
swarm/storage/localstore: add gc metrics, disable flaky test
6 years ago
3e9ba57669
swarm/storage: improve instrumentation
...
swarm/storage/localstore: fix broken metric (#1373 )
p2p/protocols: count different messages (#1374 )
cmd/swarm: disable snapshot create test due to constant flakes (#1376 )
swarm/network: remove redundant goroutine (#1377 )
6 years ago
7f753461ca
swarm/pss: disable failing handshake test
6 years ago
8802b9ce7f
swarm-smoke: add syncDelay flag
...
swarm/network: add want delay timer to syncing (#1367 )
swarm/network: synchronise peer.close() (#1369 )
6 years ago
ad6c39012f
swarm: push tags integration - request flow
...
swarm/api: integrate tags to count chunks being split and stored
swarm/api/http: integrate tags in middleware for HTTP `POST` calls and assert chunks being calculated and counted correctly
swarm: remove deprecated and unused code, add swarm hash to DoneSplit signature, remove calls to the api client from the http package
6 years ago
3030893a21
swarm/network: update syncing
6 years ago
12240baf61
swarm/chunk: add tags data type
...
* swarm/chunk: add tags backend to chunk package
6 years ago
f8eb8fe64c
cmd/swarm-smoke: check if chunks are at most prox host
...
swarm/network: measure how many chunks a node delivers (#1358 )
6 years ago
a1cd7e6e92
p2p/protocols, swarm/network: fix resource leak with p2p teardown
6 years ago
c1213bd00c
swarm: LocalStore metrics
...
* swarm/shed: remove metrics fields from DB struct
* swarm/schunk: add String methods to modes
* swarm/storage/localstore: add metrics and traces
* swarm/chunk: unknown modes without spaces in String methods
* swarm/storage/localstore: remove bin number from pull subscription metrics
* swarm/storage/localstore: add resetting time metrics and code improvements
6 years ago
993b145f25
swarm/storage/localstore: fix export db.Put signature
...
cmd/swarm/swarm-smoke: improve smoke tests (#1337 )
swarm/network: remove dead code (#1339 )
swarm/network: remove FetchStore and SyncChunkStore in favor of NetStore (#1342 )
6 years ago
996755c4a8
cmd/swarm, swarm: LocalStore storage integration
6 years ago
3873a7314d
swarm/network: fix data races in TestInitialPeersMsg test ( #19490 )
...
* swarm/network: fix data races in TestInitialPeersMsg test
* swarm/network: add Kademlia.Saturation method with lock
* swarm/network: add Hive.Peer method to safely retrieve a bzz peer
* swarm/network: remove duplicate comment
* p2p/testing: prevent goroutine leak in ProtocolTester
* swarm/network: fix data race in newBzzBaseTesterWithAddrs
* swarm/network: fix goroutone leaks in testInitialPeersMsg
* swarm/network: raise number of peer check attempts in testInitialPeersMsg
* swarm/network: use Hive.Peer in Hive.PeerInfo function
* swarm/network: reduce the scope of mutex lock in newBzzBaseTesterWithAddrs
* swarm/storage: disable TestCleanIndex with race detector
6 years ago
d9403690ec
swarm/pss: Fix flaky TestProxNetwork ( #19471 )
6 years ago
78ec90717a
swarm/version: bump version due to Geth hotfix release
6 years ago
73fc65bb5b
swarm/storage/feed: add context handling/cancellation to Swarm Feeds lookup, fix bad hint lookup bug ( #19353 )
...
* swarm/storage/feed/lookup: Add context handling/forwarding
* swarm/storage/feed/lookup: Add test to catch bad hint
* swarm/storage/feed/lookup: Added context cancellation test
6 years ago
26b50e3ebe
cmd/swarm: fix resource leaks in tests ( #19443 )
...
* swarm/api: fix file descriptor leak in NewTestSwarmServer
Swarm storage (localstore) was not closed. That resulted a
"too many open files" error if `TestClientUploadDownloadRawEncrypted`
was run with `-count 1000`.
* cmd/swarm: speed up StartNewNodes() by parallelization
Reduce cluster startup time from 13s to 7s.
* swarm/api: disable flaky TestClientUploadDownloadRawEncrypted with -race
* swarm/storage: disable flaky TestLDBStoreCollectGarbage (-race)
With race detection turned on the disabled cases often fail with:
"ldbstore_test.go:535: expected surplus chunk 150 to be missing, but got no error"
* cmd/swarm: fix process leak in TestACT and TestSwarmUp
Each test run we start 3 nodes, but we did not terminate them. So
those 3 nodes continued eating up 1.2GB (3.4GB with -race) after test
completion.
6b6c4d1c27
6 years ago
b7dd225179
swarm/version: bump Swarm due to Geth hotfix release
6 years ago
5fc5971438
swarm/version: bump version due to Geth-only hotfix release
6 years ago
5b947c5004
swarm/version: bump version due to Geth maintenance release
6 years ago
0529015091
swarm/network: hive bug: needed shallow peers are not sent to nodes beyond connection's proximity order ( #19326 )
...
* swarm/network: fix hive bug not sending shallow peers
- hive bug: needed shallow peers were not sent to nodes beyond connection's proximity order
- add extensive protocol exchange tests for initial subPeersMsg-peersMsg exchange
- modify bzzProtocolTester to allow pregenerated overlay addresses
* swarm/network: attempt to fix hive persistance test
* swarm/network: fix TestHiveStatePersistance (#1320 )
* swarm/network: remove trace lines from the hive persistance test
* address PR review comments
* swarm/network: address PR comments on TestInitialPeersMsg
* eliminate *testing.T argument from bzz/hive protocoltesters
* add sorting (only runs in test code) on peersMsg payload
* add random (0 to MaxPeersPerPO) peers for each po
* add extra peers closer to pivot than control
6 years ago
2f5b6cb442
swarm/network: Use different privatekey for bzz overlay in sim ( #19313 )
...
* cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter
* cmd/p2p/swarm: Remove comments + config.Enode nomarshal
* p2p/simulations: Remove superfluous error check
* p2p/simulation: Move init enode comment
* swarm, p2p/simulations, cmd/swarm: Use nodekey in binary record sign
* swarm/network, swarm/pss: Dervice bzzkey
* swarm/pss: Remove unused function
* swarm/network: Store swarm private key in simulation bucket
* swarm/pss: Shorten TextProxNetwork shortrunning test timeout
* swarm/pss: Increase prox test timeout
* swarm/pss: Increase timeout slightly on shortrunning proxtest
* swarm/network: Simplify bucket instantiation in servicectx func
* p2p/simulations: Tcpport -> udpport
* swarm/network, swarm/pss: Simplify + correct lock in servicefunc sim
* swarm/network: Cleanup after rebase on extract swarm enode new
* p2p/simulations, swarm/network: Make exec disc test pass
* swarm/network: Prune ye olde comment
* swarm/pss: Correct revised bzzkey method call
* swarm/network: Clarify comment about privatekey generation data
* swarm/pss: Fix syntax errors after rebase
* swarm/network: Rename misleadingly named method
(amend commit to trigger ci - attempt 5)
6 years ago
8d04154691
p2p/simulations: wait until all connections are recreated when uploading snapshot ( #19312 )
...
* swarm/network/simulation: test cases refactored
* swarm/pss: minor refactoring
* swarm/simulation: UploadSnapshot updated
* swarm/network: style fix
* swarm/pss: bugfix
6 years ago
09924cbcaa
cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter ( #19309 )
...
* cmd/swarm, p2p, swarm: Enable ENR in binary/execadapter
* cmd/p2p/swarm: Remove comments + config.Enode nomarshal
* p2p/simulations: Remove superfluous error check
* p2p/simulation: Move init enode comment
* swarm/api: Check error in config test
* swarm, p2p/simulations, cmd/swarm: Use nodekey in binary record sign
* cmd/swarm: Make nodekey available for swarm api config
6 years ago
baded64d88
swarm/network: measure time of messages in priority queue ( #19250 )
6 years ago
6e401792ce
swarm/pss: negihbourhood addressing simulation tests ( #19278 )
...
* swarm/pss: fixed bug in pss.process, test added
* swarm/pss: test case updated
* swarm/pss: WaitTillSnapshotRecreated() func added
* swarm/pss: snapshot test updated
* swarm/pss: WaitTillSnapshotLoaded() fixed
* swarm/pss: gofmt applied
* swarm/pss: refactoring, file renamed
* swarm/pss: input data fixed
* swarm/pss: race condition fixed
* swarm/pss: test timeout increased
* swarm/pss: eliminated the global variables
* swarm/pss: tests added
* swarm/pss: comments added
* swarm/pss: comment fixed
* swarm/pss: refactored according to review
* swarm/pss: style fix
* swarm/pss: increased timeout
6 years ago
f180981273
swarm/shed: add vector uint64 field ( #19279 )
6 years ago
4b4f03ca37
swarm, p2p: Prerequities for ENR replacing handshake ( #19275 )
...
* swarm/api, swarm/network, p2p/simulations: Prerequisites for handshake remove
* swarm, p2p: Add full sim node configs for protocoltester
* swarm/network: Make stream package pass tests
* swarm/network: Extract peer and addr types out of protocol file
* p2p, swarm: Make p2p/protocols tests pass + rename types.go
* swarm/network: Deactivate ExecAdapter test until binary ENR prep
* swarm/api: Remove comments
* swarm/network: Uncomment bootnode record load
6 years ago
1a3e25e4c1
swarm: tracing improvements ( #19249 )
6 years ago
9a58a9b91a
swarm/storage/localstore: global batch write lock ( #19245 )
...
* swarm/storage/localstore: most basic database
* swarm/storage/localstore: fix typos and comments
* swarm/shed: add uint64 field Dec and DecInBatch methods
* swarm/storage/localstore: decrement size counter on ModeRemoval update
* swarm/storage/localstore: unexport modeAccess and modeRemoval
* swarm/storage/localstore: add WithRetrievalCompositeIndex
* swarm/storage/localstore: add TestModeSyncing
* swarm/storage/localstore: fix test name
* swarm/storage/localstore: add TestModeUpload
* swarm/storage/localstore: add TestModeRequest
* swarm/storage/localstore: add TestModeSynced
* swarm/storage/localstore: add TestModeAccess
* swarm/storage/localstore: add TestModeRemoval
* swarm/storage/localstore: add mock store option for chunk data
* swarm/storage/localstore: add TestDB_pullIndex
* swarm/storage/localstore: add TestDB_gcIndex
* swarm/storage/localstore: change how batches are written
* swarm/storage/localstore: add updateOnAccess function
* swarm/storage/localhost: add DB.gcSize
* swarm/storage/localstore: update comments
* swarm/storage/localstore: add BenchmarkNew
* swarm/storage/localstore: add retrieval tests benchmarks
* swarm/storage/localstore: accessors redesign
* swarm/storage/localstore: add semaphore for updateGC goroutine
* swarm/storage/localstore: implement basic garbage collection
* swarm/storage/localstore: optimize collectGarbage
* swarm/storage/localstore: add more garbage collection tests cases
* swarm/shed, swarm/storage/localstore: rename IndexItem to Item
* swarm/shed: add Index.CountFrom
* swarm/storage/localstore: persist gcSize
* swarm/storage/localstore: remove composite retrieval index
* swarm/shed: IterateWithPrefix and IterateWithPrefixFrom Index functions
* swarm/storage/localstore: writeGCSize function with leveldb batch
* swarm/storage/localstore: unexport modeSetRemove
* swarm/storage/localstore: update writeGCSizeWorker comment
* swarm/storage/localstore: add triggerGarbageCollection function
* swarm/storage/localstore: call writeGCSize on DB Close
* swarm/storage/localstore: additional comment in writeGCSizeWorker
* swarm/storage/localstore: add MetricsPrefix option
* swarm/storage/localstore: fix a typo
* swamr/shed: only one Index Iterate function
* swarm/storage/localstore: use shed Iterate function
* swarm/shed: pass a new byte slice copy to index decode functions
* swarm/storage/localstore: implement feed subscriptions
* swarm/storage/localstore: add more subscriptions tests
* swarm/storage/localsore: add parallel upload test
* swarm/storage/localstore: use storage.MaxPO in subscription tests
* swarm/storage/localstore: subscription of addresses instead chunks
* swarm/storage/localstore: lock item address in collectGarbage iterator
* swarm/storage/localstore: fix TestSubscribePull to include MaxPO
* swarm/storage/localstore: improve subscriptions
* swarm/storage/localstore: add TestDB_SubscribePull_sinceAndUntil test
* swarm/storage/localstore: adjust pull sync tests
* swarm/storage/localstore: remove writeGCSizeDelay and use literal
* swarm/storage/localstore: adjust subscriptions tests delays and comments
* swarm/storage/localstore: add godoc package overview
* swarm/storage/localstore: fix a typo
* swarm/storage/localstore: update package overview
* swarm/storage/localstore: remove repeated index change
* swarm/storage/localstore: rename ChunkInfo to ChunkDescriptor
* swarm/storage/localstore: add comment in collectGarbageWorker
* swarm/storage/localstore: replace atomics with mutexes for gcSize and tests
* swarm/storage/localstore: protect addrs map in pull subs tests
* swarm/storage/localstore: protect slices in push subs test
* swarm/storage/localstore: protect chunks in TestModePutUpload_parallel
* swarm/storage/localstore: fix a race in TestDB_updateGCSem defers
* swarm/storage/localstore: remove parallel flag from tests
* swarm/storage/localstore: fix a race in testDB_collectGarbageWorker
* swarm/storage/localstore: remove unused code
* swarm/storage/localstore: add more context to pull sub log messages
* swarm/storage/localstore: BenchmarkPutUpload and global lock option
* swarm/storage/localstore: pre-generate chunks in BenchmarkPutUpload
* swarm/storage/localstore: correct useGlobalLock in collectGarbage
* swarm/storage/localstore: fix typos and update comments
* swarm/storage/localstore: update writeGCSize comment
* swarm/storage/localstore: global batch write lock
* swarm/storage/localstore: remove global lock option
* swarm/storage/localstore: simplify DB.Close
6 years ago
bb55b0fb53
swarm/storage: add comparison towards leveldb.ErrNotFound ( #19243 )
...
* swarm/storage: add comparison towards leveldb.ErrNotFound
* swarm/storage: wrap leveldb ErrNotFound
6 years ago
2cfe0bed9f
swarm: fix relationship between spans in open tracing ( #19236 )
...
* swarm/network: propagate span with ctx
* swarm/network: try to stop stream.send.request spans on time
* swarm/storage: add chunk ref as a log to netstore.fetcher span
6 years ago
a6e5c6a2cc
swarm/storage/localstore: fix synchronization in TestDB_gcSize ( #19235 )
6 years ago
eb199f1fc2
swarm: localstore hasser ( #19230 )
6 years ago
d45f8d1880
swarm/network: remove *WithServer tests from stream package ( #19223 )
...
These tests never run as the build tag excluded them from the CI
execution. As a results the (dead) code got out of sync with other
parts of Swarm and now they would not even compile. => Removed.
resolves ethersphere/go-ethereum#1238
6 years ago
a87776a5fe
swarm/network/stream: Fix flaky tests in GetSubscriptionsRPC test ( #19227 )
...
* swarm/network/stream: fixed timing issues
* swarm/network/stream: only count first iteration of subscriptions
* swarm/network/stream/: fix linter errors
6 years ago
81ed700157
Enable longrunning tests to run ( #19208 )
...
* p2p/simulations: increased snapshot load timeout for debugging
* swarm/network/stream: less nodes for snapshot longrunning tests
* swarm/network: fixed longrunning tests
* swarm/network/stream: store kademlia in bucket
* swarm/network/stream: disabled healthy check in delivery tests
* swarm/network/stream: longer SyncUpdateDelay for longrunning tests
* swarm/network/stream: more debug output
* swarm/network/stream: reduced longrunning snapshot tests to 64 nodes
* swarm/network/stream: don't WaitTillHealthy in SyncerSimulation
* swarm/network/stream: cleanup for PR
6 years ago
216bd2ceba
swarm/storage/localstore: fix testDB_collectGarbageWorker data race ( #19206 )
6 years ago
f9aa1cd21f
Revert "swarm/network: Use actual remote peer ip in underlay ( #19137 )" ( #19193 )
...
This reverts commit 460d206f30
6 years ago
b797dd07d2
swarm/shed, swarm/storage/localstore: add LastPullSubscriptionChunk ( #19190 )
...
* swarm/shed, swarm/storage/localstore: add LastPullSubscriptionChunk
* swarm/shed: fix comments
* swarm/shed: fix TestIncByteSlice test
* swarm/storage/localstore: fix TestDB_LastPullSubscriptionChunk
6 years ago
4e9230ea7a
swarm: enable p2p/discovery and disable dynamic dialling ( #19189 )
6 years ago
994326ba00
swarm: new snapshot files ( #19185 )
6 years ago
62d9d63858
swarm/network: WIP consider all nodes for healthy iteration ( #19155 )
...
* swarm/network: WIP consider all nodes for healthy iteration
* swarm/network/simulation: extend TestWaitTillHealthy to really check kads are healthy
* cmd/swarm/swarm-snapshot: fixed bugs in snapshot creation binary
* swarm/network/simulation: addressed PR comments
* swarm/network/simulation: defer sim.Clsoe()
* swarm/network/simulation: fixed wrong sim.Close()
* swarm/network/simulation: addressed PR comments
* cmd/swarm/swarm-snapshot: reducing default to 8 nodes, more to 4
* cmd/swarm/swarm-snapshot: extended timeout to 3 mins, or 256 nodes snapshot times out
* swarm/network/simulation: More PR comments
6 years ago
Anton Evangelatov
Elad
holisticode
lash
Janoš Guljaš
gluk256
Péter Szilágyi
Javier Peletier
Ferenc Szabo
Viktor Trón