Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve docker compose coverage #2320

Merged
merged 69 commits into from
Mar 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
806a13e
feat: generate plan yamls (#2177)
tedim52 Mar 13, 2024
3c25691
feat: support package nodes in enclave builder ui (#2283)
tedim52 Mar 13, 2024
e20fb32
feat: port guessing based on port number for docker transpiler (#2281)
h4ck3rk3y Mar 13, 2024
94ec6e5
chore(main): release 0.88.6 (#2278)
kurtosisbot Mar 13, 2024
af950b5
build(deps): Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in…
dependabot[bot] Mar 13, 2024
9c1b09b
build(deps): Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in…
dependabot[bot] Mar 13, 2024
5cf05f7
build(deps): Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in…
dependabot[bot] Mar 13, 2024
2f93b11
build(deps): Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in…
dependabot[bot] Mar 13, 2024
530d49c
build(deps): Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in…
dependabot[bot] Mar 13, 2024
ca6de4d
build(deps): Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in…
dependabot[bot] Mar 13, 2024
9d22932
elts see if this owrks
h4ck3rk3y Mar 14, 2024
345e361
ran tidy
h4ck3rk3y Mar 14, 2024
895f36d
some more intelligent edit
h4ck3rk3y Mar 14, 2024
502f82a
fix abs path
tedim52 Mar 14, 2024
5468fb5
fix: remove env vars with password (#2300)
tedim52 Mar 14, 2024
de9ca2d
this should work
h4ck3rk3y Mar 14, 2024
830cc8e
this should work
h4ck3rk3y Mar 14, 2024
52a0205
revert excess
h4ck3rk3y Mar 14, 2024
3bdc94e
fix validator
h4ck3rk3y Mar 14, 2024
f17dd63
another bug fix
h4ck3rk3y Mar 14, 2024
2ee3e36
this should be okay
h4ck3rk3y Mar 14, 2024
e9d4b61
another fix
h4ck3rk3y Mar 14, 2024
6954907
chore(main): release 0.88.7 (#2301)
kurtosisbot Mar 14, 2024
81f7385
improvemnt
h4ck3rk3y Mar 14, 2024
df4b2f4
build(deps): Bump follow-redirects from 1.15.4 to 1.15.6 in /enclave-…
dependabot[bot] Mar 14, 2024
0669f1f
lets try this
h4ck3rk3y Mar 14, 2024
553d7e2
more logs
h4ck3rk3y Mar 14, 2024
7f1b597
last try
h4ck3rk3y Mar 14, 2024
919dff0
this was the problem
h4ck3rk3y Mar 14, 2024
38495d3
this was the problem
h4ck3rk3y Mar 14, 2024
c0522c1
does this work
h4ck3rk3y Mar 15, 2024
e1c4dde
handle secrests key
tedim52 Mar 15, 2024
5fbe401
feat: Add active cancel at period end payment subscription status (#2…
laurentluce Mar 15, 2024
2ed2028
this seems okay
h4ck3rk3y Mar 15, 2024
9e598e4
-c works
h4ck3rk3y Mar 15, 2024
be0269d
this is good;
h4ck3rk3y Mar 15, 2024
273861c
fix: the package replace logic when using tag, branch or commit (#2299)
leoporoli Mar 15, 2024
fe6fbe4
this is a work in progress
h4ck3rk3y Mar 18, 2024
2c2a286
paranthesis
h4ck3rk3y Mar 18, 2024
e24a441
added an echo for testing
h4ck3rk3y Mar 18, 2024
4e5076c
chore(main): release 0.88.8 (#2308)
kurtosisbot Mar 18, 2024
55b8a18
cleaned a large function
h4ck3rk3y Mar 18, 2024
b769d26
some refactoring
h4ck3rk3y Mar 18, 2024
5947105
less fragile
h4ck3rk3y Mar 18, 2024
c860b34
quote fixing
h4ck3rk3y Mar 18, 2024
511a8d0
handle quoting later
h4ck3rk3y Mar 18, 2024
a441216
fix logic;
h4ck3rk3y Mar 18, 2024
0c58631
fix: `ImageBuildSpec` `build_file` feature (#2313)
tedim52 Mar 18, 2024
9332940
handle service level env file
tedim52 Mar 18, 2024
e9c2e6e
remove underscores from service names for rfc1035 naming compliance
tedim52 Mar 18, 2024
d115a0f
Merge branch 'gyani/file-mount' into tedi/composecoverage
tedim52 Mar 18, 2024
ee17542
fix image logic
tedim52 Mar 19, 2024
8a48df9
provide root user if caps add
tedim52 Mar 19, 2024
330024a
Update docker_compose_transpiler.go
h4ck3rk3y Mar 19, 2024
cf43fd6
address tmp case
tedim52 Mar 19, 2024
87116d7
force the move
tedim52 Mar 19, 2024
a67d299
if dir, just upload files
tedim52 Mar 19, 2024
83f89fe
use container name if set
tedim52 Mar 20, 2024
cdf1a33
enable getting on disk path for repositories
tedim52 Mar 20, 2024
b372214
Merge branch 'main' into gyani/file-mount
tedim52 Mar 20, 2024
0d85ae1
clean up
tedim52 Mar 21, 2024
c9cf9bf
update tests
tedim52 Mar 21, 2024
811ade0
Merge branch 'main' into tedi/composecoverage
tedim52 Mar 21, 2024
a57a654
update tests
tedim52 Mar 21, 2024
bbc1c64
Merge branch 'gyani/file-mount' into tedi/composecoverage
tedim52 Mar 21, 2024
9369f8f
address comments
tedim52 Mar 21, 2024
010b17e
Merge branch 'main' into tedi/composecoverage
tedim52 Mar 21, 2024
3a768a2
Merge branch 'main' into tedi/composecoverage
tedim52 Mar 22, 2024
3b8f911
check nil image inspect config
tedim52 Mar 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ package user_service_functions

import (
"context"
"fmt"
"strconv"
"strings"
"sync"

Expand Down Expand Up @@ -539,6 +541,7 @@ func createStartServiceOperation(
memoryAllocationMegabytes := serviceConfig.GetMemoryAllocationMegabytes()
privateIPAddrPlaceholder := serviceConfig.GetPrivateIPAddrPlaceholder()
user := serviceConfig.GetUser()
filesToBeMoved := serviceConfig.GetFilesToBeMoved()

// We replace the placeholder value with the actual private IP address
privateIPAddrStr := privateIpAddr.String()
Expand All @@ -552,6 +555,17 @@ func createStartServiceOperation(
envVars[key] = strings.Replace(envVars[key], privateIPAddrPlaceholder, privateIPAddrStr, unlimitedReplacements)
}

// TODO clean this hack up
// this path will only be hit if `files_to_be_moved` is set in Starlark; which is a hidden property
// used by compose transpilation
if len(filesToBeMoved) > 0 {
tedim52 marked this conversation as resolved.
Show resolved Hide resolved
var err error
cmdArgs, entrypointArgs, err = getUpdatedEntrypointAndCmdFromFilesToBeMoved(ctx, dockerManager, containerImageName, cmdArgs, entrypointArgs, filesToBeMoved)
if err != nil {
return nil, stacktrace.Propagate(err, "an error occurred while handling files for compose")
}
}

volumeMounts := map[string]string{}
shouldDeleteVolumes := true
if filesArtifactsExpansion != nil {
Expand Down Expand Up @@ -753,6 +767,50 @@ func createStartServiceOperation(
}
}

// TODO - clean this up this is super janky, a way to handle compose & volume
func getUpdatedEntrypointAndCmdFromFilesToBeMoved(ctx context.Context, dockerManager *docker_manager.DockerManager, containerImageName string, cmdArgs []string, entrypointArgs []string, filesToBeMoved map[string]string) ([]string, []string, error) {
concatenatedFilesToBeMoved := []string{}
for source, destination := range filesToBeMoved {
// TODO improve this; the first condition handles files the other folders
concatenatedFilesToBeMoved = append(concatenatedFilesToBeMoved, fmt.Sprintf("mv %v %v", source, destination))
}

concatenatedFilesToBeMovedAsStr := strings.Join(concatenatedFilesToBeMoved, " && ")
originalEntrypointArgs, originalCmdArgs, err := dockerManager.GetEntryPointAndCommand(ctx, containerImageName)
if err != nil {
return nil, nil, stacktrace.Propagate(err, "an error occurred fetching data about image '%v'", containerImageName)
}

// we do this replacement as we want to keep the original command args as they are not overwritten
// it might be that none of the two are set
if len(cmdArgs) == 0 && len(originalCmdArgs) > 0 {
cmdArgs = originalCmdArgs
}
if len(entrypointArgs) == 0 && len(originalEntrypointArgs) > 0 {
entrypointArgs = originalEntrypointArgs
}

entryPointArgsAsStr := quoteAndJoinArgs(entrypointArgs)
cmdArgsAsStr := quoteAndJoinArgs(cmdArgs)

if len(cmdArgs) > 0 {
if len(entrypointArgs) > 0 {
cmdArgs = []string{"-c", concatenatedFilesToBeMovedAsStr + " && " + entryPointArgsAsStr + " " + cmdArgsAsStr}
} else {
cmdArgs = []string{"-c", concatenatedFilesToBeMovedAsStr + " && " + cmdArgsAsStr}
}
} else {
if len(entrypointArgs) > 0 {
cmdArgs = []string{"-c", concatenatedFilesToBeMovedAsStr + " && " + entryPointArgsAsStr}
} else {
// no entrypoint and no command; this shouldn't really happen
cmdArgs = []string{"-c", concatenatedFilesToBeMovedAsStr}
}
}
entrypointArgs = []string{"/bin/sh"}
return cmdArgs, entrypointArgs, nil
}

// Ensure that provided [privatePorts] and [publicPorts] are one to one by checking:
// - There is a matching publicPort for every portID in privatePorts
// - There are the same amount of private and public ports
Expand Down Expand Up @@ -845,3 +903,11 @@ func registerUserServices(

return successfulRegistrations, failedRegistrations, nil
}

func quoteAndJoinArgs(args []string) string {
var quotedArgs []string
for _, arg := range args {
quotedArgs = append(quotedArgs, strconv.Quote(arg))
}
return strings.Join(quotedArgs, " ")
}
Original file line number Diff line number Diff line change
Expand Up @@ -1660,6 +1660,17 @@ func (manager *DockerManager) getImagePlatform(ctx context.Context, imageName st
return imageInspect.Architecture, nil
}

func (manager *DockerManager) GetEntryPointAndCommand(ctx context.Context, imageName string) ([]string, []string, error) {
imageInspect, _, err := manager.dockerClient.ImageInspectWithRaw(ctx, imageName)
if err != nil {
return nil, nil, stacktrace.Propagate(err, "an error occurred while running image inspect on image '%v'", imageName)
}
if imageInspect.Config == nil {
return nil, nil, stacktrace.NewError("image inspect config was empty, can't geet entrypoint or cmd: %v", imageInspect)
}
return imageInspect.Config.Entrypoint, imageInspect.Config.Cmd, nil
tedim52 marked this conversation as resolved.
Show resolved Hide resolved
}

/*
Creates a Docker-Container-To-Host Port mapping, defining how a Container's JSON RPC and service-specific ports are
mapped to the host ports.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@ package service

import (
"encoding/json"
"github.com/kurtosis-tech/kurtosis/container-engine-lib/lib/backend_interface/objects/image_download_mode"

"github.com/kurtosis-tech/kurtosis/container-engine-lib/lib/backend_interface/objects/image_build_spec"
"github.com/kurtosis-tech/kurtosis/container-engine-lib/lib/backend_interface/objects/image_download_mode"
"github.com/kurtosis-tech/kurtosis/container-engine-lib/lib/backend_interface/objects/image_registry_spec"
"github.com/kurtosis-tech/kurtosis/container-engine-lib/lib/backend_interface/objects/nix_build_spec"
"github.com/kurtosis-tech/kurtosis/container-engine-lib/lib/backend_interface/objects/port_spec"
Expand Down Expand Up @@ -70,6 +69,8 @@ type privateServiceConfig struct {
NodeSelectors map[string]string

ImageDownloadMode image_download_mode.ImageDownloadMode

FilesToBeMoved map[string]string
}

func CreateServiceConfig(
Expand Down Expand Up @@ -123,6 +124,7 @@ func CreateServiceConfig(
Tolerations: tolerations,
NodeSelectors: nodeSelectors,
ImageDownloadMode: imageDownloadMode,
FilesToBeMoved: map[string]string{},
}
return &ServiceConfig{internalServiceConfig}, nil
}
Expand Down Expand Up @@ -217,6 +219,14 @@ func (serviceConfig *ServiceConfig) GetNodeSelectors() map[string]string {
return serviceConfig.privateServiceConfig.NodeSelectors
}

func (serviceConfig *ServiceConfig) SetFilesToBeMoved(filesToBeMoved map[string]string) {
serviceConfig.privateServiceConfig.FilesToBeMoved = filesToBeMoved
}

func (serviceConfig *ServiceConfig) GetFilesToBeMoved() map[string]string {
return serviceConfig.privateServiceConfig.FilesToBeMoved
}

func (serviceConfig *ServiceConfig) UnmarshalJSON(data []byte) error {

// Suppressing exhaustruct requirement because we want an object with zero values
Expand Down
Loading
Loading