Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System test fixes for new simplified read-pool #277

Merged
merged 13 commits into from
Jun 6, 2022
Merged

System test fixes for new simplified read-pool #277

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e8c37ac
Test fix for new readpool
iamrz1 May 25, 2022
bf154e6
Merge branch 'master' into user-read-pools
iamrz1 May 25, 2022
9954248
Fixed integration test for new read-pool
iamrz1 May 27, 2022
bc7394b
Merge branch 'master' into user-read-pools
iamrz1 May 27, 2022
e09066d
Further test fix for read-pool
iamrz1 May 27, 2022
c57b360
Read pool test fix
iamrz1 May 27, 2022
29a0a76
Fix max_read_price test for read
iamrz1 May 27, 2022
1913bdf
Merge branch 'master' into user-read-pools
iamrz1 Jun 1, 2022
a083558
Revert read lock parameters to createParams based string
iamrz1 Jun 2, 2022
217540f
Fixed bool value in create params
iamrz1 Jun 2, 2022
008566d
Merge branch 'master' of github.com:0chain/system_test into user-read…
iamrz1 Jun 5, 2022
10024cf
Fix Download File Without read-lock for new readpool
iamrz1 Jun 6, 2022
8cf9cf7
Merge branch 'master' into user-read-pools
dabasov Jun 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions internal/cli/model/model.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,8 @@ type Blobber struct {
}

type ReadPoolInfo struct {
Id string `json:"id"`
Balance int64 `json:"balance"`
ExpireAt int64 `json:"expire_at"`
AllocationId string `json:"allocation_id"`
Blobber []Blobber `json:"blobbers"`
Locked bool `json:"locked"`
OwnerBalance int64 `json:"owner_balance"`
VisitorBalance int64 `json:"visitor_balance"`
}

type WritePoolInfo struct {
Expand Down
32 changes: 11 additions & 21 deletions tests/cli_tests/flaky___broken_scenarios_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,16 @@ package cli_tests
import (
"encoding/json"
"fmt"
climodel "github.com/0chain/system_test/internal/cli/model"
cliutils "github.com/0chain/system_test/internal/cli/util"
"github.com/stretchr/testify/require"
"os"
"path"
"path/filepath"
"regexp"
"strconv"
"strings"
"testing"
"time"

climodel "github.com/0chain/system_test/internal/cli/model"
cliutils "github.com/0chain/system_test/internal/cli/util"
"github.com/stretchr/testify/require"
)

/*
Expand Down Expand Up @@ -384,18 +382,13 @@ func Test___FlakyBrokenScenarios(t *testing.T) {
require.Equal(t, 1, len(meta.Collaborators), "Collaborator must be added in file collaborators list")
require.Equal(t, collaboratorWallet.ClientID, meta.Collaborators[0].ClientID, "Collaborator must be added in file collaborators list")

output, err = readPoolLock(t, configPath, createParams(map[string]interface{}{
"allocation": allocationID,
"tokens": 0.4,
"duration": "1h",
}), true)
output, err = readPoolLock(t, configPath, "--tokens 0.4", true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The createParams logic above is consistent with the rest of the test codebase, why has it been removed in favour of literal strings?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to my testing, boolean flag doesn't work with parameter map, and read pool lock command has boolean flag. hence all the call to lock read-pool has been changed to look similar in case we need to use the parameter more in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can pass through a boolean with
output, err = readPoolLock(t, configPath, createParams(map[string]interface{}{
"booleanName": "",
}), true)
or
output, err = readPoolLock(t, configPath, createParams(map[string]interface{}{
"booleanName": "true",
}), true)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see how we do it for the commit flag for example
https://github.com/0chain/system_test/pull/282/files#diff-b64b1f0b1e26c3819b06b4df6e3961b113082b9e744f7b4ea499371cf8459c89R60

Copy link
Contributor Author

@iamrz1 iamrz1 Jun 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would work with bool values that assume the default value to be false. I have added a fix in createParams that would work with boolean values directly, so we don't have to use blank string for false values.
Also reversed all the strings back to params using createParams.
PTAL.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, much tidier

require.Nil(t, err, "Tokens could not be locked", strings.Join(output, "\n"))
require.Len(t, output, 1, "Unexpected number of output lines", strings.Join(output, "\n"))
require.Equal(t, "locked", output[0])

readPool := getReadPoolInfo(t, allocationID)
require.Len(t, readPool, 1, "Read pool must exist")
require.Equal(t, ConvertToValue(0.4), readPool[0].Balance, "Read Pool balance must be equal to locked amount")
readPool := getReadPoolInfo(t)
require.Equal(t, ConvertToValue(0.4), readPool.OwnerBalance, "Read Pool balance must be equal to locked amount")

output, err = downloadFileForWallet(t, collaboratorWalletName, configPath, createParams(map[string]interface{}{
"allocation": allocationID,
Expand All @@ -408,15 +401,12 @@ func Test___FlakyBrokenScenarios(t *testing.T) {
expectedOutput := fmt.Sprintf("Status completed callback. Type = application/octet-stream. Name = %s", filepath.Base(localpath))
require.Equal(t, expectedOutput, output[1], "Unexpected output", strings.Join(output, "\n"))

// Wait for read markers to be redeemed
cliutils.Wait(t, 5*time.Second)

readPool = getReadPoolInfo(t, allocationID)
require.Len(t, readPool, 1, "Read pool must exist")
// expected download cost times to the number of blobbers
expectedPoolBalance := ConvertToValue(0.4) - int64(len(readPool[0].Blobber))*expectedDownloadCost
require.InEpsilon(t, expectedPoolBalance, readPool[0].Balance, 0.000001, "Read Pool balance must be equal to (initial balace-download cost)")
t.Logf("Expected Read Pool Balance: %v\nActual Read Pool Balance: %v", expectedPoolBalance, readPool[0].Balance)
expectedPoolBalance := ConvertToValue(0.4) - expectedDownloadCost

updatedReadPool, err := getReadPoolUpdate(t, readPool, 5)
require.NoError(t, err)
require.Equal(t, expectedPoolBalance, updatedReadPool.OwnerBalance, "Read Pool balance must be equal to (initial balance-download cost)")
})

t.Run("Tokens should move from write pool balance to challenge pool acc. to expected upload cost", func(t *testing.T) {
Expand Down
18 changes: 5 additions & 13 deletions tests/cli_tests/miner_fees_payment_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,17 +175,12 @@ func TestMinerFeesPayment(t *testing.T) {
output, err = executeFaucetWithTokens(t, configPath, 1.0)
require.Nil(t, err, "error executing faucet", strings.Join(output, "\n"))

allocationId := setupAllocation(t, configPath)
//allocationId := setupAllocation(t, configPath)
iamrz1 marked this conversation as resolved.
Show resolved Hide resolved

startBlock := getLatestFinalizedBlock(t)

fee := 0.1
output, err = readPoolLock(t, configPath, createParams(map[string]interface{}{
"allocation": allocationId,
"tokens": 0.5,
"duration": "1m",
"fee": fee,
}), true)
output, err = readPoolLock(t, configPath, fmt.Sprintf("--tokens %v --fee %v", 0.5, fee), true)
require.Nil(t, err, "error locking read pool tokens", strings.Join(output, "\n"))

lockTimer := time.NewTimer(time.Minute)
Expand All @@ -201,21 +196,18 @@ func TestMinerFeesPayment(t *testing.T) {
areMinerFeesPaidCorrectly := verifyMinerFeesPayment(t, &block, expectedMinerFee)
require.True(t, areMinerFeesPaidCorrectly, "Test Failed due to transfer from MinerSC to generator miner not found")

output, err = readPoolInfo(t, configPath, allocationId)
output, err = readPoolInfo(t, configPath)
require.Nil(t, err, "error fetching read pool", strings.Join(output, "\n"))
require.Len(t, output, 1)
readPool := []climodel.ReadPoolInfo{}
readPool := climodel.ReadPoolInfo{}
err = json.Unmarshal([]byte(output[0]), &readPool)
require.Nil(t, err, "error unmarshalling read pool", strings.Join(output, "\n"))

<-lockTimer.C

startBlock = getLatestFinalizedBlock(t)

output, err = readPoolUnlock(t, configPath, createParams(map[string]interface{}{
"pool_id": readPool[0].Id,
"fee": fee,
}), true)
output, err = readPoolUnlock(t, configPath, createParams(map[string]interface{}{}), true)
require.Nil(t, err, "error unlocking read pool", strings.Join(output, "\n"))

cliutils.Wait(t, 30*time.Second)
Expand Down
56 changes: 31 additions & 25 deletions tests/cli_tests/zboxcli_collaborator_system_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,18 +118,13 @@ func TestCollaborator(t *testing.T) {
require.Len(t, meta.Collaborators, 1, "Collaborator must be added in file collaborators list")
require.Equal(t, collaboratorWallet.ClientID, meta.Collaborators[0].ClientID, "Collaborator must be added in file collaborators list")

output, err = readPoolLock(t, configPath, createParams(map[string]interface{}{
"allocation": allocationID,
"tokens": 0.4,
"duration": "1h",
}), true)
output, err = readPoolLock(t, configPath, "--tokens 0.4", true)
require.Nil(t, err, "Tokens could not be locked", strings.Join(output, "\n"))
require.Len(t, output, 1, "Unexpected number of output lines", strings.Join(output, "\n"))
require.Equal(t, "locked", output[0])

readPool := getReadPoolInfo(t, allocationID)
require.Len(t, readPool, 1, "Read pool must exist")
require.Equal(t, ConvertToValue(0.4), readPool[0].Balance, "Read Pool balance must be equal to locked amount")
readPool := getReadPoolInfo(t)
require.Equal(t, ConvertToValue(0.4), readPool.OwnerBalance, "Read Pool balance must be equal to locked amount")

output, err = downloadFileForWallet(t, collaboratorWalletName, configPath, createParams(map[string]interface{}{
"allocation": allocationID,
Expand Down Expand Up @@ -265,11 +260,7 @@ func TestCollaborator(t *testing.T) {
require.Equal(t, collaboratorWallet.ClientID, meta.Collaborators[0].ClientID, "Collaborator must be added in file collaborators list")

// Lock tokens in read pool
output, err = readPoolLock(t, configPath, createParams(map[string]interface{}{
"allocation": allocationID,
"tokens": 0.4,
"duration": "1h",
}), true)
output, err = readPoolLock(t, configPath, "--tokens 0.4", true)
require.Nil(t, err, "Tokens could not be locked", strings.Join(output, "\n"))
require.Len(t, output, 1, "Unexpected number of output lines", strings.Join(output, "\n"))
require.Equal(t, "locked", output[0])
Expand Down Expand Up @@ -717,18 +708,14 @@ func TestCollaborator(t *testing.T) {
require.Len(t, meta.Collaborators, 1, "Collaborator must be added in file collaborators list")
require.Equal(t, collaboratorWallet.ClientID, meta.Collaborators[0].ClientID, "Collaborator must be added in file collaborators list")

output, err = readPoolLock(t, configPath, createParams(map[string]interface{}{
"allocation": allocationID,
"tokens": 0.4,
"duration": "1h",
}), true)
lockedTokens := 0.4
output, err = readPoolLock(t, configPath, fmt.Sprintf("--tokens %v", lockedTokens), true)
require.Nil(t, err, "Tokens could not be locked", strings.Join(output, "\n"))
require.Len(t, output, 1, "Unexpected number of output lines", strings.Join(output, "\n"))
require.Equal(t, "locked", output[0])

readPool := getReadPoolInfo(t, allocationID)
require.Len(t, readPool, 1, "Read pool must exist")
require.Equal(t, ConvertToValue(0.4), readPool[0].Balance, "Read Pool balance must be equal to locked amount")
readPool := getReadPoolInfo(t)
require.Equal(t, ConvertToValue(lockedTokens), readPool.OwnerBalance, "Read Pool balance must be equal to locked amount")

output, err = downloadFileForWallet(t, collaboratorWalletName, configPath, createParams(map[string]interface{}{
"allocation": allocationID,
Expand All @@ -742,14 +729,33 @@ func TestCollaborator(t *testing.T) {
})
}

func getReadPoolInfo(t *testing.T, allocationID string) []climodel.ReadPoolInfo {
output, err := readPoolInfo(t, configPath, allocationID)
func getReadPoolUpdate(t *testing.T, erp climodel.ReadPoolInfo, retry int) (climodel.ReadPoolInfo, error) {
if retry == 0 {
retry = 1
}
// Wait for read markers to be redeemed
for i := 0; i < retry; i++ {
readPool := getReadPoolInfo(t)
if (erp.OwnerBalance > 0 && readPool.OwnerBalance == erp.OwnerBalance) ||
(erp.VisitorBalance > 0 && readPool.VisitorBalance == erp.VisitorBalance) {
continue
}

cliutils.Wait(t, time.Second*30)
return getReadPoolInfo(t), nil
}

return erp, fmt.Errorf("no update found in readpool")
}

func getReadPoolInfo(t *testing.T) climodel.ReadPoolInfo {
output, err := readPoolInfo(t, configPath)
require.Nil(t, err, "Error fetching read pool", strings.Join(output, "\n"))
require.Len(t, output, 1)

readPool := []climodel.ReadPoolInfo{}
var readPool climodel.ReadPoolInfo
err = json.Unmarshal([]byte(output[0]), &readPool)
require.Nil(t, err, "Error unmarshalling read pool", strings.Join(output, "\n"))
require.Nil(t, err, "Error unmarshalling read pool %s", strings.Join(output, "\n"))
return readPool
}

Expand Down
17 changes: 6 additions & 11 deletions tests/cli_tests/zboxcli_download_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -650,7 +650,7 @@ func TestDownload(t *testing.T) {

// This test creates a separate wallet and allocates there, test nesting is required to create another wallet json file
t.Run("Share File from Another Wallet", func(t *testing.T) {
allocationID = setupAllocationAndReadLock(t, configPath, map[string]interface{}{
allocationID = setupAllocation(t, configPath, map[string]interface{}{
"size": 10 * 1024,
"tokens": 1,
})
Expand All @@ -670,21 +670,19 @@ func TestDownload(t *testing.T) {
output, err := shareFolderInAllocation(t, configPath, shareParam)
require.Nil(t, err, strings.Join(output, "\n"))
require.Len(t, output, 1)

authTicket, err = extractAuthToken(output[0])
require.Nil(t, err, "extract auth token failed")
require.NotEqual(t, "", authTicket, "Ticket: ", authTicket)
})

err = registerWalletAndLockReadTokens(t, configPath, allocationID)
err = registerWalletAndLockReadTokens(t, configPath, false)
require.Nil(t, err)
// Download file using auth-ticket: should work
output, err := downloadFile(t, configPath, createParams(map[string]interface{}{
"authticket": authTicket,
"localpath": "tmp/",
"rx_pay": "",
}), false)

require.Nil(t, err, strings.Join(output, "\n"))
require.Len(t, output, 2)
aggregatedOutput := strings.Join(output, " ")
Expand Down Expand Up @@ -1316,9 +1314,10 @@ func TestDownload(t *testing.T) {
"localpath": "tmp/",
}), false)
require.NotNil(t, err, strings.Join(output, "\n"))
require.Len(t, output, 3)
t.Log("len(output):", len(output))
require.Len(t, output, 5)
aggregatedOutput := strings.Join(output, " ")
require.Contains(t, aggregatedOutput, "not enough tokens")
require.Contains(t, aggregatedOutput, "Download failed")
})

t.Run("Download File using Expired Allocation Should Fail", func(t *testing.T) {
Expand Down Expand Up @@ -1398,11 +1397,7 @@ func setupAllocationAndReadLock(t *testing.T, cliConfigFilename string, extraPar
allocationID := setupAllocation(t, cliConfigFilename, extraParam)

// Lock half the tokens for read pool
output, err := readPoolLock(t, cliConfigFilename, createParams(map[string]interface{}{
"allocation": allocationID,
"tokens": tokens / 2,
"duration": "10m",
}), true)
output, err := readPoolLock(t, cliConfigFilename, fmt.Sprintf("--tokens %v", tokens/2), true)
require.Nil(t, err, strings.Join(output, "\n"))
require.Len(t, output, 1)
require.Equal(t, "locked", output[0])
Expand Down
Loading