Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

shape/parse-date function is not tolerant of null values #155

Closed
ericthorsen opened this issue Nov 3, 2020 · 2 comments
Closed

shape/parse-date function is not tolerant of null values #155

ericthorsen opened this issue Nov 3, 2020 · 2 comments
Assignees
@dchelimsky
Labels
bug Something isn't working

Comments

@ericthorsen
Copy link

Description

It appears shape/parse-date function is not tolerant of null values.
Given this json payload from aws/backup ListBackupPlans:

  "BackupPlansList": [
    {
      "AdvancedBackupSettings": null,
      "BackupPlanArn": "arn:aws:backup:us-east-2:474533704392:backup-plan:3d9ef9cb-e1eb-4aee-8d57-87f9ef9936be",
      "BackupPlanId": "3d9ef9cb-e1eb-4aee-8d57-87f9ef9936be",
      "BackupPlanName": "Minecraft-server-backup-plan",
      "CreationDate": 1.604233663729E9,
      "CreatorRequestId": null,
      "DeletionDate": null,
      "LastExecutionDate": 1.604294512369E9,
      "VersionId": "NGE2M2M2MTgtNTYyNS00ZjI0LTk4ZmQtNjMxZDMyNzA5YzQ0"
    }
  ],
  "NextToken": null
}

I would expect an edn representation to look like this:

{:NextToken nil,
 :BackupPlansList
 [{:BackupPlanArn
   "arn:aws:backup:us-east-2:474533704392:backup-plan:3d9ef9cb-e1eb-4aee-8d57-87f9ef9936be",
   :BackupPlanId "3d9ef9cb-e1eb-4aee-8d57-87f9ef9936be",
   :CreationDate #inst "2020-11-01T12:27:43.000-00:00",
   :DeletionDate nil,
   :VersionId "NGE2M2M2MTgtNTYyNS00ZjI0LTk4ZmQtNjMxZDMyNzA5YzQ0",
   :BackupPlanName "Minecraft-server-backup-plan",
   :CreatorRequestId nil,
   :LastExecutionDate #inst "2020-11-02T05:21:52.000-00:00"}]}

...but i get an exception instead.
The code in question is here src/cognitect/aws/shape.clj#L99

Possible Solution

Add a nil? check in the parse date fn

diff --git a/src/cognitect/aws/shape.clj b/src/cognitect/aws/shape.clj
index a3cfa2f..d945e73 100644
--- a/src/cognitect/aws/shape.clj
+++ b/src/cognitect/aws/shape.clj
@@ -86,7 +86,8 @@

 (defn parse-date
   [{:keys [timestampFormat]} data]
-  (cond (= "rfc822" timestampFormat)
+  (cond (nil? data) nil
+        (= "rfc822" timestampFormat)
         (util/parse-date util/rfc822-date-format data)
         (= "iso8601"timestampFormat)
         (parse-date* data

Dependencies

{:paths ["src"]
 :deps { org.clojure/clojure {:mvn/version "1.10.1"}
        com.cognitect/anomalies #:mvn {:version "0.1.12"}
        com.cognitect.aws/api       {:mvn/version "0.8.474"}
        com.cognitect.aws/endpoints {:mvn/version "1.1.11.842"}
        com.cognitect.aws/backup {:mvn/version "801.2.704.0", :aws/serviceFullName "AWS Backup"}
        }

Not sure you need this but here is a full recipe to recreate the scenario....

;; Begin repl code
(require
    '[clojure.pprint :as pp]
    '[cognitect.aws.client.api :as aws]
    '[cognitect.aws.credentials :as credentials]
    '[cognitect.aws.region :as region]
    )
                        
;Create an AWS backup client
(def backup-client
  (aws/client
   {:api :backup
    :credentials-provide (credentials/profile-credentials-provider)
    :region-provider (region/profile-region-provider)}))

; Create a vault
(aws/invoke backup-client
            {:op  :CreateBackupVault
             :request
             {:BackupVaultName "deleteThisVault"}})

; Make sure it's there
(aws/invoke backup-client {:op :ListBackupVaults})

; create a backup plan
(def plan (aws/invoke backup-client
            {:op :CreateBackupPlan
             :request
             {:BackupPlan
              {:BackupPlanName "delete-this-plan"
               :Rules
               [{:RuleName "Daily backup rule"
                 :ScheduleExpression "cron(0 2 * * ? *)"
                 :TargetBackupVaultName "deleteThisVault"
                 :Lifecycle
                 {:DeleteAfterDays 15}}]}}}))

; EXCEPTION is here: This returns nil dates which cause an exception
(aws/invoke backup-client {:op :ListBackupPlans})

;Cleanup the resources created above...
(aws/invoke backup-client {:op :DeleteBackupPlan
                           :request
                           {:BackupPlanId
                            (get plan :BackupPlanId)}})

(aws/invoke backup-client {:op :DeleteBackupVault
                           :request
                           {:BackupVaultName "deleteThisVault"}})
;; end repl

Stack traces

 :cognitect.aws.client/throwable #error {
 :cause nil
 :via
 [{:type java.lang.NullPointerException
   :message nil
   :at [java.util.regex.Matcher getTextLength "Matcher.java" 1770]}]
 :trace
 [[java.util.regex.Matcher getTextLength "Matcher.java" 1770]
  [java.util.regex.Matcher reset "Matcher.java" 416]
  [java.util.regex.Matcher <init> "Matcher.java" 253]
  [java.util.regex.Pattern matcher "Pattern.java" 1135]
  [clojure.core$re_matcher invokeStatic "core.clj" 4856]
  [clojure.core$re_matches invokeStatic "core.clj" 4886]
  [clojure.core$re_matches invoke "core.clj" 4886]
  [cognitect.aws.shape$parse_date invokeStatic "shape.clj" 99]
  [cognitect.aws.shape$parse_date invoke "shape.clj" 87]
  [cognitect.aws.shape$eval23704$fn__23705 invoke "shape.clj" 162]
  [clojure.lang.MultiFn invoke "MultiFn.java" 234]
  [cognitect.aws.shape$eval23689$fn__23690$fn__23691 invoke "shape.clj" 147]
  [clojure.core.protocols$iter_reduce invokeStatic "protocols.clj" 49]
  [clojure.core.protocols$fn__8138 invokeStatic "protocols.clj" 75]
  [clojure.core.protocols$fn__8138 invoke "protocols.clj" 75]
  [clojure.core.protocols$fn__8088$G__8083__8101 invoke "protocols.clj" 13]
  [clojure.core$reduce invokeStatic "core.clj" 6828]
  [clojure.core$reduce invoke "core.clj" 6810]
  [cognitect.aws.shape$eval23689$fn__23690 invoke "shape.clj" 143]
  [clojure.lang.MultiFn invoke "MultiFn.java" 234]
  [cognitect.aws.shape$handle_list$fn__23672 invoke "shape.clj" 123]
  [clojure.core$mapv$fn__8445 invoke "core.clj" 6912]
  [clojure.lang.PersistentVector reduce "PersistentVector.java" 343]
  [clojure.core$reduce invokeStatic "core.clj" 6827]
  [clojure.core$mapv invokeStatic "core.clj" 6903]
  [clojure.core$mapv invoke "core.clj" 6903]
  [cognitect.aws.shape$handle_list invokeStatic "shape.clj" 123]
  [cognitect.aws.shape$handle_list invoke "shape.clj" 121]
  [cognitect.aws.shape$eval23700$fn__23701 invoke "shape.clj" 158]
  [clojure.lang.MultiFn invoke "MultiFn.java" 234]
  [cognitect.aws.shape$eval23689$fn__23690$fn__23691 invoke "shape.clj" 147]
  [clojure.core.protocols$iter_reduce invokeStatic "protocols.clj" 49]
  [clojure.core.protocols$fn__8138 invokeStatic "protocols.clj" 75]
  [clojure.core.protocols$fn__8138 invoke "protocols.clj" 75]
  [clojure.core.protocols$fn__8088$G__8083__8101 invoke "protocols.clj" 13]
  [clojure.core$reduce invokeStatic "core.clj" 6828]
  [clojure.core$reduce invoke "core.clj" 6810]
  [cognitect.aws.shape$eval23689$fn__23690 invoke "shape.clj" 143]
  [clojure.lang.MultiFn invoke "MultiFn.java" 234]
  [cognitect.aws.shape$json_parse invokeStatic "shape.clj" 207]
  [cognitect.aws.shape$json_parse invoke "shape.clj" 204]
  [cognitect.aws.protocols.rest$parse_body invokeStatic "rest.clj" 247]
  [cognitect.aws.protocols.rest$parse_body invoke "rest.clj" 235]
  [cognitect.aws.protocols.rest$parse_http_response invokeStatic "rest.clj" 260]
  [cognitect.aws.protocols.rest$parse_http_response invoke "rest.clj" 249]
  [cognitect.aws.protocols.rest_json$eval26752$fn__26753 invoke "rest_json.clj" 61]
  [clojure.lang.MultiFn invoke "MultiFn.java" 239]
  [cognitect.aws.client$handle_http_response invokeStatic "client.clj" 48]
  [cognitect.aws.client$handle_http_response invoke "client.clj" 43]
  [cognitect.aws.client$send_request$fn__24616$state_machine__19004__auto____24643$fn__24646 invoke "client.clj" 111]
  [cognitect.aws.client$send_request$fn__24616$state_machine__19004__auto____24643 invoke "client.clj" 107]
  [clojure.core.async.impl.ioc_macros$run_state_machine invokeStatic "ioc_macros.clj" 978]
  [clojure.core.async.impl.ioc_macros$run_state_machine invoke "ioc_macros.clj" 977]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invokeStatic "ioc_macros.clj" 982]
  [clojure.core.async.impl.ioc_macros$run_state_machine_wrapped invoke "ioc_macros.clj" 980]
  [clojure.core.async.impl.ioc_macros$take_BANG_$fn__19022 invoke "ioc_macros.clj" 991]
  [clojure.core.async.impl.channels.ManyToManyChannel$fn__13008$fn__13009 invoke "channels.clj" 95]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1130]
  [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 630]
  [clojure.core.async.impl.concurrent$counted_thread_factory$reify__12877$fn__12878 invoke "concurrent.clj" 29]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 832]]}}
@dchelimsky
Copy link
Contributor

Thanks for the detailed report @ericthorsen. Fixed here. Will release today or tomorrow.

@dchelimsky dchelimsky self-assigned this Nov 3, 2020
@dchelimsky dchelimsky added the bug Something isn't working label Nov 3, 2020
@dchelimsky
Copy link
Contributor

Released in aws-api-0.8.484

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dchelimsky dchelimsky

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dchelimsky @ericthorsen