ID automatically generated by PostgreSQL becomes undefined #4751

hayato8823hayato · 2020-02-26T13:04:40Z

A situation similar to #3749

Steps to reproduce

lb4 datasource
lb4 discover
lb4 repository

? Please select the datasource TestDbDatasource
? Select the model(s) you want to generate a repository TestTable
? Please select the repository base class DefaultCrudRepository (Legacy juggler bridge)

lb4 controller

? Controller class name: Test
Controller Test will be created in src/controllers/test.controller.ts

? What kind of controller would you like to generate? REST Controller with CRUD functions
? What is the name of the model to use with this CRUD repository? TestTable
? What is the name of your CRUD repository? TestTableRepository
? What is the name of ID property? testId
? What is the type of your ID? number
? Is the id omitted when creating a new instance? Yes
? What is the base HTTP path name of the CRUD operations? /test-tables

Modify model

@model({
  settings: {
    idInjection: false,
    postgresql: {schema: 'public', table: 'testTable'},
  },
})
export class TestTable extends Entity {
  @property({
    type: 'number',
    required: false, // true -> false
    scale: 0,
    id: 1,
    postgresql: {
      columnName: 'testId',
      dataType: 'bigint',
      dataLength: null,
      dataPrecision: null,
      dataScale: 0,
      nullable: 'NO',
    },
  })
  testId: number;

  @property({
    type: 'string',
    required: true,
    length: 100,
    postgresql: {
      columnName: 'testContent',
      dataType: 'character varying',
      dataLength: 100,
      dataPrecision: null,
      dataScale: null,
      nullable: 'NO',
    },
  })
  testContent: string;

  // Define well-known properties here

  // Indexer property to allow additional data
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  [prop: string]: any;

  constructor(data?: Partial<TestTable>) {
    super(data);
  }
}

Current Behavior

Do the following

const test = await this.testTableRepository.create({testContent: "test"});
console.log(test);

Result

TestTable { testId: undefined, testContent: 'test' }

The instance is being created in the database, but it somehow failed to map to the property.

Expected Behavior

I want it to look like the following

TestTable { testId: 1, testContent: 'test' }

Additional information

The following is a guess

I think the problem is that it is toLowerCase() in the function here.
https://github.com/strongloop/loopback-connector-postgresql/blob/v3.9.0/lib/postgresql.js#L369

After rewriting as below, it worked.

PostgreSQL.prototype.dbName = function(name) {
  if (!name) {
    return name;
  }
  // PostgreSQL default to lowercase names
  // return name.toLowerCase();
  return name;
};

PostgreSQL.prototype.getInsertedId
https://github.com/strongloop/loopback-connector-postgresql/blob/v3.9.0/lib/postgresql.js#L553
SQLConnector.prototype.idColumn
https://github.com/strongloop/loopback-connector/blob/v4.4.0/lib/sql.js#L423
PostgreSQL.prototype.dbName
https://github.com/strongloop/loopback-connector-postgresql/blob/v3.9.0/lib/postgresql.js#L369

lb4 --version

lb4 --version
@loopback/cli version: 1.30.1

@loopback/* dependencies:
  - @loopback/authentication: ^3.3.3
  - @loopback/boot: ^1.7.4
  - @loopback/build: ^3.1.1
  - @loopback/context: ^2.1.1
  - @loopback/core: ^1.12.4
  - @loopback/metadata: ^1.4.1
  - @loopback/openapi-spec-builder: ^1.3.1
  - @loopback/openapi-v3: ^2.0.0
  - @loopback/repository-json-schema: ^1.12.2
  - @loopback/repository: ^1.19.1
  - @loopback/rest: ^2.0.0
  - @loopback/testlab: ^1.10.3
  - @loopback/docs: ^2.11.0
  - @loopback/example-hello-world: ^1.2.25
  - @loopback/example-log-extension: ^1.2.25
  - @loopback/example-rpc-server: ^1.2.25
  - @loopback/example-todo: ^2.0.0
  - @loopback/example-soap-calculator: ^1.7.7
  - @loopback/service-proxy: ^1.3.17
  - @loopback/http-caching-proxy: ^1.3.0
  - @loopback/http-server: ^1.5.4
  - @loopback/example-todo-list: ^2.0.0
  - @loopback/dist-util: ^0.4.0
  - @loopback/rest-explorer: ^1.4.10
  - @loopback/eslint-config: ^5.0.3
  - @loopback/example-express-composition: ^1.10.4
  - @loopback/example-greeter-extension: ^1.3.25
  - @loopback/booter-lb3app: ^1.3.12
  - @loopback/example-lb3-application: ^1.1.26
  - @loopback/example-greeting-app: ^1.2.12
  - @loopback/example-context: ^1.3.2
  - @loopback/repository-tests: ^0.10.1
  - @loopback/extension-health: ^0.2.17
  - @loopback/authorization: ^0.4.10
  - @loopback/rest-crud: ^0.6.6
  - @loopback/security: ^0.1.13
  - @loopback/authentication-passport: ^1.1.3
  - @loopback/example-metrics-prometheus: ^0.1.7
  - @loopback/extension-metrics: ^0.1.6
  - @loopback/model-api-builder: ^1.1.4
  - @loopback/extension-logging: ^0.1.0

Related Issues

Acceptance Criteria

if the property has defined its column name, Postgres shouldn't use the default naming convention (lowercase), should keep as it is
besides Postgres, Oracle has the same issue ( uppercase as default case)
add tests for discover

The text was updated successfully, but these errors were encountered:

agnes512 · 2020-02-26T14:57:13Z

@raw-ham Hi, the model definition looks legit to me. I think the problem is caused by id auto-generation. Could you check what's your default value for the identifier of table testTable with query:

SELECT column_name, column_default, data_type FROM information_schema.columns WHERE table_schema = 'public' and table_name = 'testTable';

If the id property in your table is not autogenerated (e.g default value of identifier is SERIAL), the result of the operation

const test = await this.testTableRepository.create({testContent: "test"});

will be testId: undefined.

There are three ways to fix it:
1.

set required to true and include value of id in POST request.
if you use the controller, the setting needs to be modified:

? Controller class name: Test
Controller Test will be created in src/controllers/test.controller.ts

? What kind of controller would you like to generate? REST Controller with CRUD functions
? What is the name of the model to use with this CRUD repository? TestTable
? What is the name of your CRUD repository? TestTableRepository
? What is the name of ID property? testId
? What is the type of your ID? number
? Is the id omitted when creating a new instance? No // this should be no instead
? What is the base HTTP path name of the CRUD operations? /test-tables

ALTER the testTable to set default value of the identity column to SERIAL

Define your model first, then use npm run migration ( notice that migration and discover are different):

@model()
export class testTable extends Entity {
  @property({
    type: 'number',
    id: true,
    required: false // not necessary, but it cannot be true
    generated: true, // this will set the default value of the identifier for you
    postgresql: {  // this setting allows you to have different names for model and db column
      columnName: 'testid', // this needs to be in lowercase
      dataType: 'integer',
      dataLength: null,
      dataPrecision: null,
      dataScale: 0,
      nullable: 'NO',
    },
  })
  testId?: number;

  // other props..

  constructor(data?: Partial<TestTable>) {
    super(data);
  }
}

hayato8823hayato · 2020-02-26T16:28:29Z

@agnes512 Thanks for your answer

Could you check what's your default value for the identifier of table testTable with query:

There are three ways to fix it:

I can't do either fix.
Because I uses an already running DB.

No problem occurs if the DB column name is lowercase or snake case.
(No problem when using lb4 discover)

I think the problem is that idColName becomes lowercase.
https://github.com/strongloop/loopback-connector-postgresql/blob/v3.9.0/lib/postgresql.js#L554

Or is there a restriction that ID columns can only be used in lowercase or snakecase?

I use Google Translate.
Sorry if it was difficult to read.

agnes512 · 2020-02-26T20:53:43Z

@raw-ham I can reproduce the issue, and it only happens in Postgres. I think it's still because of the identifier issue. And weird thing is that the workaround doesn't work anymore D:

(workaround:

export class TestTable extends Entity {
  @property({
    type: 'number',
    required: false, 
    scale: 0,
    id: 1,
    postgresql: {
      columnName: 'testId',
      dataType: 'bigint',
      dataLength: null,
      dataPrecision: null,
      dataScale: 0,
      nullable: 'NO',
    },
  })
  testid: number; // in lowercase
...

)

The only way to make it work now is to set the identifier column name to lowercase for Postgres, which seems a unpleasant bug to me. issue #3343 is for column name mapping. It doesn't solve this specific issue in Postgres. I will discuss with the team and hopefully fix it soon. Thanks for reporting the bug!

hayato8823hayato · 2020-02-27T00:17:58Z

@agnes512

The only way to make it work now is to set the identifier column name to lowercase for Postgres,

Yes, there is no other solution. So I am in trouble.

I will discuss with the team and hopefully fix it soon. Thanks for reporting the bug!

Thank you very much! Please comment if you need more information.

agnes512 · 2020-02-27T01:56:58Z

Debug note:
juggler-model.definition: ModelDefinition.prototype.idColumnName -(testId)> juggler-DataSource.prototype.idColumnName -(testId) > connector-SQLConnector.prototype.idColumn -( it calls dbName function of each connector)> postgres-connector-postgresql.js line 280 -> testid

jannyHou · 2020-02-27T19:14:10Z

For code change in loopback-connector/sql/idColumnName(), it will affect other sql connectors, but if that function is overridden in the other connector, it's fine.

agnes512 · 2020-03-04T18:12:37Z

@raw-ham a fix is released in [email protected], the identifier column should be testId
with definition:

  @property({
    type: 'number',
    required: false,
    scale: 0,
    id: 1,
    postgresql: {
      columnName: 'testId',
      dataType: 'bigint',
      dataLength: null,
      dataPrecision: null,
      dataScale: 0,
      nullable: 'NO',
    },
  })
  testId: number;

Could you try it out to see if it works on your end? thanks

hayato8823hayato · 2020-03-08T08:42:35Z

@agnes512

I confirmed that it was solved in my project.

Thank you for correspondence!

hayato8823hayato added the bug label Feb 26, 2020

achrinza added the db:PostgreSQL Topics specific to PostgreSQL label Feb 26, 2020

agnes512 self-assigned this Feb 26, 2020

agnes512 added the Model Discovery Issues related to discovery of model definitions from (SQL) databases label Feb 26, 2020

This was referenced Feb 27, 2020

[DO NOT MERGE] feat(docs): march milestone #4594

Closed

fix how connector gets column name for id property loopbackio/loopback-connector#171

Merged

agnes512 mentioned this issue Feb 28, 2020

Monthly Milestone - March 2020 👾⏳🐨 #4767

Closed

23 tasks

hayato8823hayato closed this as completed Mar 8, 2020

agnes512 added this to the Mar 2020 milestone Mar 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ID automatically generated by PostgreSQL becomes undefined #4751

ID automatically generated by PostgreSQL becomes undefined #4751

hayato8823hayato commented Feb 26, 2020 •

edited by agnes512

Loading

agnes512 commented Feb 26, 2020

hayato8823hayato commented Feb 26, 2020 •

edited

Loading

agnes512 commented Feb 26, 2020 •

edited

Loading

hayato8823hayato commented Feb 27, 2020

agnes512 commented Feb 27, 2020

jannyHou commented Feb 27, 2020

agnes512 commented Mar 4, 2020

hayato8823hayato commented Mar 8, 2020

ID automatically generated by PostgreSQL becomes undefined #4751

ID automatically generated by PostgreSQL becomes undefined #4751

Comments

hayato8823hayato commented Feb 26, 2020 • edited by agnes512 Loading

Steps to reproduce

Current Behavior

Expected Behavior

Additional information

The following is a guess

lb4 --version

Related Issues

Acceptance Criteria

agnes512 commented Feb 26, 2020

hayato8823hayato commented Feb 26, 2020 • edited Loading

agnes512 commented Feb 26, 2020 • edited Loading

hayato8823hayato commented Feb 27, 2020

agnes512 commented Feb 27, 2020

jannyHou commented Feb 27, 2020

agnes512 commented Mar 4, 2020

hayato8823hayato commented Mar 8, 2020

hayato8823hayato commented Feb 26, 2020 •

edited by agnes512

Loading

hayato8823hayato commented Feb 26, 2020 •

edited

Loading

agnes512 commented Feb 26, 2020 •

edited

Loading