Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AWS install doesn't work #11873

Open
alberttwong opened this issue Nov 16, 2024 · 3 comments
Open

AWS install doesn't work #11873

alberttwong opened this issue Nov 16, 2024 · 3 comments
Labels
bug Bug report

Comments

@alberttwong
Copy link
Contributor

I'm following the AWS install at https://datahubproject.io/docs/deploy/aws/. I'm at the helm install step at https://datahubproject.io/docs/deploy/kubernetes.
I'm running the install but the install doesn't work. it's in pending status. any ideas?

helm install prerequisites datahub/datahub-prerequisites
NAME: prerequisites
LAST DEPLOYED: Fri Nov 15 15:52:22 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1

❯ kubectl get pods
NAME                           READY   STATUS    RESTARTS   AGE
elasticsearch-master-0         0/1     Pending   0          14m
prerequisites-kafka-broker-0   0/1     Pending   0          14m
prerequisites-mysql-0          0/1     Pending   0          14m
prerequisites-zookeeper-0      0/1     Pending   0          14m
@alberttwong alberttwong added the bug Bug report label Nov 16, 2024
@alberttwong
Copy link
Contributor Author

#11872

@alberttwong
Copy link
Contributor Author

you need to do this. add on the CSI EBS driver and add the correct role to the eks node group. https://datahubspace.slack.com/archives/C029A3M079U/p1722437477880419?thread_ts=1722288192.887179&cid=C029A3M079U.

@alberttwong
Copy link
Contributor Author

not sure if this running correctly

❯ kubectl get pods
NAME                                             READY   STATUS      RESTARTS   AGE
datahub-acryl-datahub-actions-788d98d6b4-mhxd9   1/1     Running     0          18m
datahub-datahub-frontend-574896c955-xktcs        1/1     Running     0          18m
datahub-datahub-gms-f68bb8964-zchvw              1/1     Running     0          18m
datahub-elasticsearch-setup-job-jxh8h            0/1     Completed   0          23m
datahub-kafka-setup-job-6hz76                    0/1     Completed   0          23m
datahub-mysql-setup-job-r5sgk                    0/1     Completed   0          20m
datahub-nocode-migration-job-5ldkx               0/1     Completed   0          16m
datahub-nocode-migration-job-htfpt               0/1     Error       0          18m
datahub-system-update-ggr76                      0/1     Completed   0          20m
datahub-system-update-nonblk-nsvzz               0/1     Completed   0          14m
elasticsearch-master-0                           1/1     Running     0          33m
prerequisites-kafka-broker-0                     1/1     Running     0          59m
prerequisites-mysql-0                            1/1     Running     0          29m
prerequisites-zookeeper-0                        1/1     Running     0          59m

2024-11-16 00:35:31,906 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - Skipping Step 1/6: RemoveAspectV2TableStep...
2024-11-16 00:35:31,906 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - Executing Step 2/6: GMSQualificationStep...
java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.connect0(Native Method)
	at java.base/sun.nio.ch.Net.connect(Net.java:579)
	at java.base/sun.nio.ch.Net.connect(Net.java:568)
	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:593)
	at java.base/java.net.Socket.connect(Socket.java:633)
	at java.base/java.net.Socket.connect(Socket.java:583)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:183)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:533)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:638)
	at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:281)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1308)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1241)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1127)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1056)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1686)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1610)
	at com.linkedin.datahub.upgrade.common.steps.GMSQualificationStep.lambda$executable$0(GMSQualificationStep.java:87)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeStepInternal(DefaultUpgradeManager.java:129)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:80)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:49)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.execute(DefaultUpgradeManager.java:37)
	at com.linkedin.datahub.upgrade.UpgradeCli.run(UpgradeCli.java:93)
	at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:790)
	at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:83)
	at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
	at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:88)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789)
	at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:774)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:342)
	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:149)
	at com.linkedin.datahub.upgrade.UpgradeCliApplication.main(UpgradeCliApplication.java:40)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53)
	at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58)
2024-11-16 00:35:33,004 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - ERROR: Cannot connect to GMSat http://host datahub-datahub-gms port 8080. Make sure GMS is on the latest version and is running at that host before starting the migration.
java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.connect0(Native Method)
	at java.base/sun.nio.ch.Net.connect(Net.java:579)
	at java.base/sun.nio.ch.Net.connect(Net.java:568)
	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:593)
	at java.base/java.net.Socket.connect(Socket.java:633)
	at java.base/java.net.Socket.connect(Socket.java:583)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:183)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:533)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:638)
	at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:281)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1308)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1241)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1127)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1056)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1686)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1610)
	at com.linkedin.datahub.upgrade.common.steps.GMSQualificationStep.lambda$executable$0(GMSQualificationStep.java:87)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeStepInternal(DefaultUpgradeManager.java:129)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:80)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:49)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.execute(DefaultUpgradeManager.java:37)
	at com.linkedin.datahub.upgrade.UpgradeCli.run(UpgradeCli.java:93)
	at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:790)
	at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:83)
	at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
	at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:88)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789)
	at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:774)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:342)
	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:149)
	at com.linkedin.datahub.upgrade.UpgradeCliApplication.main(UpgradeCliApplication.java:40)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53)
	at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58)
2024-11-16 00:35:34,030 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - ERROR: Cannot connect to GMSat http://host datahub-datahub-gms port 8080. Make sure GMS is on the latest version and is running at that host before starting the migration.
java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.connect0(Native Method)
	at java.base/sun.nio.ch.Net.connect(Net.java:579)
	at java.base/sun.nio.ch.Net.connect(Net.java:568)
	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:593)
	at java.base/java.net.Socket.connect(Socket.java:633)
	at java.base/java.net.Socket.connect(Socket.java:583)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:183)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:533)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:638)
	at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:281)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1308)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1241)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1127)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1056)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1686)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1610)
	at com.linkedin.datahub.upgrade.common.steps.GMSQualificationStep.lambda$executable$0(GMSQualificationStep.java:87)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeStepInternal(DefaultUpgradeManager.java:129)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:80)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.executeInternal(DefaultUpgradeManager.java:49)
	at com.linkedin.datahub.upgrade.impl.DefaultUpgradeManager.execute(DefaultUpgradeManager.java:37)
	at com.linkedin.datahub.upgrade.UpgradeCli.run(UpgradeCli.java:93)
	at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:790)
	at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:83)
	at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
	at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:88)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789)
	at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:774)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:342)
	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:149)
	at com.linkedin.datahub.upgrade.UpgradeCliApplication.main(UpgradeCliApplication.java:40)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53)
	at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58)
2024-11-16 00:35:35,052 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - ERROR: Cannot connect to GMSat http://host datahub-datahub-gms port 8080. Make sure GMS is on the latest version and is running at that host before starting the migration.
2024-11-16 00:35:35,053 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - Failed Step 2/6: GMSQualificationStep. Failed after 2 retries.
2024-11-16 00:35:35,053 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - Exiting upgrade NoCodeDataMigration with failure.
2024-11-16 00:35:35,053 [main] INFO  c.l.d.u.impl.DefaultUpgradeReport:15 - Upgrade NoCodeDataMigration completed with result FAILED. Exiting...
2024-11-16 00:35:35,066 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.AbstractNettyClient:249 - Shutdown requested
2024-11-16 00:35:35,068 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.AbstractNettyClient:252 - Shutting down
2024-11-16 00:35:35,076 [EbeanHook] INFO  io.ebean.datasource:122 - DataSource [gmsEbeanDatabaseConfig] shutdown min[2] max[50] free[2] busy[0] waiting[0] highWaterMark[1] waitCount[0] hitCount[4] meanAcquireNanos[16000] maxAcquireMicros[22]  psc[hit:0 miss:0 put:0 rem:0]
2024-11-16 00:35:35,092 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.ChannelPoolManagerImpl:152 - Shutting down 0 connection pools
2024-11-16 00:35:35,095 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.ChannelPoolManagerImpl:162 - All connection pools shutdown
2024-11-16 00:35:35,095 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.ChannelPoolManagerImpl:119 - All connection pools shut down, closing all channels
2024-11-16 00:35:35,198 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.ChannelPoolManagerImpl:152 - Shutting down 0 connection pools
2024-11-16 00:35:35,199 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.ChannelPoolManagerImpl:162 - All connection pools shutdown
2024-11-16 00:35:35,199 [SpringApplicationShutdownHook] INFO  c.l.r.t.h.c.c.ChannelPoolManagerImpl:119 - All

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Bug report
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@alberttwong