Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate type for object header #652

Open
cthulhu-rider opened this issue Dec 6, 2024 · 8 comments
Open

Separate type for object header #652

cthulhu-rider opened this issue Dec 6, 2024 · 8 comments
Labels
enhancement Improving existing functionality I2 Regular impact S2 Regular significance U4 Nothing urgent
Milestone

Comments

@cthulhu-rider
Copy link
Contributor

Is your feature request related to a problem? Please describe.

currently, object provides Object structure type, but no Header one. This type would be helpful for HEAD ops (Client, node storage, etc.). Now such ops return instances of the Object type with unset payload field. I see following drawbacks in this approach:

  1. header is always requested by object ID, and the resulting instance has the one. This is redundant now
  2. header encoding becomes more tricky. This leads to additions like https://pkg.go.dev/github.com/nspcc-dev/neofs-sdk-go/object#Object.MarshalHeaderJSON, which is not rly bad, but could be provided apriori. For binary encoding, https://pkg.go.dev/github.com/nspcc-dev/neofs-sdk-go/object#Object.CutPayload is used, but it adds ID to the result while these BLOBS are usually stored by ID key itself
  3. entity appears in the protocol and in different system parts, it is quite natural for it to have a type
  4. in code, the header is not distinguished from the object, although it is a narrower type. Because of this, comments have to be added, which weakens the type system

Describe the solution you'd like

type Header struct { /* container, owner, ... */ }
type Object struct {
	Header
	id        oid.ID
	signature neofscrypto.Signature
	payload   []byte
}
func (*Client) HeadObject(...) (object.Header, neofscrypto.Signature, error)

Additional context

iirc @carpawell had questions why there is no header type. I couldnt find any issue, pls share if smth already exists

@cthulhu-rider cthulhu-rider added discussion Open discussion of some problem feature Completely new functionality labels Dec 6, 2024
@carpawell
Copy link
Member

I always did not understand why there is no header while IMO it is a natural understanding for our HEAD operation and objects in general: every object has a header and every object has a payload. Sometimes we need only a header and vice versa. I have not opened any issues about it.

@roman-khimov
Copy link
Member

Yeah, we need this type, passing object.Object and doing CutPayload() tricks is very confusing, you never know what you have in code: a complete object or header only. But we need it to follow proto spec as well, signature is a part of the header IIRC.

@roman-khimov roman-khimov added enhancement Improving existing functionality U4 Nothing urgent S2 Regular significance I2 Regular impact and removed discussion Open discussion of some problem feature Completely new functionality labels Dec 9, 2024
@roman-khimov roman-khimov added this to the v1.0.0-rc13 milestone Dec 9, 2024
@cthulhu-rider
Copy link
Contributor Author

@roman-khimov
Copy link
Member

Header, SignedHeader, Object?

@cthulhu-rider
Copy link
Contributor Author

Header, SignedHeader, Object?

SignedHeader includes ID here?

@roman-khimov
Copy link
Member

It's so tempting for me to make it a method with cached internal value (transaction.Transaction). We need to evaluate real use cases, I know we have a number of uses for SignedHeader (maybe even more so than for Header alone).

@cthulhu-rider
Copy link
Contributor Author

cached values ​​are often a source of errors. I would stick with simple tuples with open logic

real use cases

https://pkg.go.dev/github.com/nspcc-dev/neofs-sdk-go/object#Object.MarshalHeaderJSON is what i remember. https://github.com/nspcc-dev/neo-go/blob/5f92da21fa43e4fe80fb965c78d5d244b6003ffd/pkg/services/oracle/neofs/neofs.go#L185. The getObjHeader could return object.Header

various tuples can be used:

and i remember no usecase where we need (ID, signature, header) tuple

finally, for now i see HeaderHeaderWithSignatureObject type system

@cthulhu-rider
Copy link
Contributor Author

cthulhu-rider commented Dec 9, 2024

it must be noted that transition to the heading types must be carried out carefully. For example, CutPayload returns an object. So, https://github.com/nspcc-dev/neofs-node/blob/918307b0739feb086416e667497c676170e0c9ca/pkg/local_object_storage/metabase/put.go#L186 means that metabase stores (ID, signature, header) tuples. Not rly good to store storage key in the value, but it is what it is, updated code must take this into account. Normally, metabase could store HeaderWithSignature

ALSO!!!: do not forget about field numbers. Object and HeaderWithSignature have different ones. Field nums are essential for Marshal and any protobuf functionality

cthulhu-rider added a commit that referenced this issue Dec 16, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 16, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 16, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 16, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 17, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

API client unit tests and AIO integration ones PASS, therefore, there is
no more way to track more regressions for now. If problems arise in
apps when updating the lib, fixes will be added later before the major
release.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 17, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

API client unit tests and AIO integration ones PASS, therefore, there is
no more way to track more regressions for now. If problems arise in
apps when updating the lib, fixes will be added later before the major
release.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 17, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

API client unit tests and AIO integration ones PASS, therefore, there is
no more way to track more regressions for now. If problems arise in
apps when updating the lib, fixes will be added later before the major
release.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 17, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

API client unit tests and AIO integration ones PASS, therefore, there is
no more way to track more regressions for now. If problems arise in
apps when updating the lib, fixes will be added later before the major
release.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 17, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

API client unit tests and AIO integration ones PASS, therefore, there is
no more way to track more regressions for now. If problems arise in
apps when updating the lib, fixes will be added later before the major
release.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 18, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

API client unit tests and AIO integration ones PASS, therefore, there is
no more way to track more regressions for now. If problems arise in
apps when updating the lib, fixes will be added later before the major
release.

Closes #214. Refs #226. Refs #270. Closes #452. Closes #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 25, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
 1. All exported elements that depended on neofs-api-go types are
 permanently removed. They could not be excluded via a temporary
 deprecation mark as this would cause a conflict with importing ''*.pb.go'
 files. Such elements were not recommended for use in advance, so most
 of the updated software will not suffer from breakages.

 2. Request/response signing and verification functions have been added
 to `crypto` package A. They are used by the API client and will also be
 used by https://github.com/nspcc-dev/neofs-node.

 3. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
 while protobuf lib decodes them into `~int32`. In addition, during
 encoding/decoding, all unknown enum values were set to zero, which
 could lead to data loss. This commit fixes both issues. Separating the
 fix from the refactoring would have required adding artificial buggy
 code, it was decided not to generate it.

 4. Test binaries/JSONs/signatures that followed the erroneous
 behavior of p.3 are fixed.

 5. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
 `ProtoMessage` method replacing `WriteToV2`, taking into account the
 proposal from #532 and existing use cases, now dynamically allocates
 the message structure.

API client unit tests and AIO integration ones PASS, therefore, there is
no more way to track more regressions for now. If problems arise in
apps when updating the lib, fixes will be added later before the major
release.

Closes #214. Refs #226. Refs #270. Closes #452. Refs #532. Closes #551
(it's almost impossible to do now). Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 25, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
1. All exported elements that depended on neofs-api-go types are
permanently removed. They could not be excluded via a temporary
deprecation mark as this would cause a conflict with importing ''*.pb.go'
files. Such elements were not recommended for use in advance, so most
of the updated software will not suffer from breakages.

2. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
while protobuf lib decodes them into `~int32`. In addition, during
encoding/decoding, all unknown enum values were set to zero, which
could lead to data loss. This commit fixes both issues. Separating the
fix from the refactoring would have required adding artificial buggy
code, it was decided not to generate it.

3. Test binaries/JSONs/signatures that followed the erroneous
behavior of p.2 are fixed.

4. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
`ProtoMessage` method replacing `WriteToV2`, taking into account the
proposal from #532 and existing use cases, now dynamically allocates
the message structure.

NOTE: `client` package is going to be changed the same way in the
following commits.

NOTE 2: some tests panic due to protobuf namespace conflict, which will
be fixed automatically after the move is complete.

Closes #214. Refs #226. Refs #532. Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
cthulhu-rider added a commit that referenced this issue Dec 26, 2024
Since 527c964, proto-generated code is
located in the current code library. Previously, the neofs-api-go
module's code was used for protocol interactions. In essence, it is a
super-layer on top of the `google.golang.org/protobuf` module with only
one feature: stable marshalling (Protocol Buffers with ascending field
order). Since it is now provided by local `proto/*` packages, there is
no longer a need for a separate module.

In addition to trimming the code base and refactoring, a bonus is the
allocation of fewer intermediate Go objects, which will have a
beneficial effect on runtime.

Although most of the changes are refactorings, the following changes
have been applied:
1. All exported elements that depended on neofs-api-go types are
permanently removed. They could not be excluded via a temporary
deprecation mark as this would cause a conflict with importing ''*.pb.go'
files. Such elements were not recommended for use in advance, so most
of the updated software will not suffer from breakages.

2. `neofs-api-go` a bug with interpreting enums as unsigned numbers,
while protobuf lib decodes them into `~int32`. In addition, during
encoding/decoding, all unknown enum values were set to zero, which
could lead to data loss. This commit fixes both issues. Separating the
fix from the refactoring would have required adding artificial buggy
code, it was decided not to generate it.

3. Test binaries/JSONs/signatures that followed the erroneous
behavior of p.2 are fixed.

4. While replacement of `ReadFromV2` methods called `FromProtoMessage`,
`ProtoMessage` method replacing `WriteToV2`, taking into account the
proposal from #532 and existing use cases, now dynamically allocates
the message structure.

NOTE: `client` package is going to be changed the same way in the
following commits.

NOTE 2: some tests panic due to protobuf namespace conflict, which will
be fixed automatically after the move is complete.

Closes #214. Refs #226. Refs #532. Fixes #606. Refs #652.

Signed-off-by: Leonard Lyubich <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improving existing functionality I2 Regular impact S2 Regular significance U4 Nothing urgent
Projects
None yet
Development

No branches or pull requests

3 participants