ocfl

package module
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 10, 2025 License: MIT Imports: 36 Imported by: 8

README

OCFL for Go

godocs doi

This is an implementation of the Oxford Common File Layout for Go. The module can be used in Go programs to support operations on OCFL storage roots and objects. It supports the local file system or s3 storage backends. Several complete example programs are included.

[!WARNING]
The API is under heavy development and will have constant breaking changes.

This module is used in the following projects:

Development

Requires Go >= v1.24.

Documentation

Overview

This module is an implementation of the Oxford Common File Layout (OCFL) specification. The top-level package provides version-independent functionality. The ocflv1 package provides the bulk of implementation.

Index

Constants

View Source 
const (
	NamasteTypeObject = "ocfl_object" // type string for OCFL Object declaration
	NamasteTypeRoot   = "ocfl"        // type string for OCFL Storage Root declaration
)
View Source 
const (
	// HasNamaste indicates that an object root directory includes a NAMASTE
	// object declaration file
	HasNamaste objectStateFlag = 1 << iota
	// HasInventory indicates that an object root includes an "inventory.json"
	// file
	HasInventory
	// HasSidecar indicates that an object root includes an "inventory.json.*"
	// file (the inventory sidecar).
	HasSidecar
	// HasExtensions indicates that an object root includes a directory named
	// "extensions"
	HasExtensions
	//HasLogs indicates that an object root includes a directory named "logs"
	HasLogs
)
View Source 
const (
	// package version
	Version = "0.11.0"

	Spec1_0 = Spec("1.0")
	Spec1_1 = Spec("1.1")
)

Variables

View Source 
var (
	ErrNamasteNotExist = fmt.Errorf("missing NAMASTE declaration: %w", fs.ErrNotExist)
	ErrNamasteContents = errors.New("invalid NAMASTE declaration contents")
	ErrNamasteMultiple = errors.New("multiple NAMASTE declarations found")
)
View Source 
var (
	ErrOCFLNotImplemented    = errors.New("unimplemented or missing version of the OCFL specification")
	ErrObjectNamasteExists   = fmt.Errorf("found existing OCFL object declaration: %w", fs.ErrExist)
	ErrObjectNamasteNotExist = fmt.Errorf("the OCFL object declaration does not exist: %w", ErrNamasteNotExist)
	ErrObjRootStructure      = errors.New("object includes invalid files or directories")
)
View Source 
var (
	ErrSpecInvalid  = errors.New("invalid OCFL spec version")
	ErrSpecNotFound = errors.New("OCFL spec file not found")
)
View Source 
var (
	ErrVNumInvalid = errors.New(`invalid version`)
	ErrVNumPadding = errors.New(`inconsistent version padding in version sequence`)
	ErrVNumMissing = errors.New(`missing version in version sequence`)
	ErrVerEmpty    = errors.New("no versions found")
)
View Source 
var (
	// ErrInventorySidecarContents indicates the inventory sidecar contents is
	// not formatted correctly.
	ErrInventorySidecarContents = errors.New("invalid contents of inventory sidecar file")
)
View Source 
var ErrLayoutUndefined = errors.New("storage root's layout is undefined")
View Source 
var ErrMapMakerExists = errors.New("path and digest exist")

ErrMapMakerExists is returned when calling Add with a path and digest that are already present in the MapMaker

View Source 
var ErrNoObjectID = errors.New("object does not exist: an explicit ID is required but was not provided")
View Source 
var ErrObjectReadOnly = errors.New("object is read-only")
View Source 
var ErrRevertUpdate = errors.New("the update has completed and cannot be reverted")

ErrRevertUpdate: can't revert an update because the update ran to completion

Functions

func ParseVNum 

func ParseVNum(v string, vn *VNum) error

ParseVNum parses string as an a VNum and sets the value referenced by vn.

func ReadInventorySidecar added in v0.9.0 

func ReadInventorySidecar(ctx context.Context, fsys ocflfs.FS, dir, alg string) (string, error)

ReadInventorySidecar reads the digest from an inventory sidecar file in dir, using the digest algorithm alg.

func ValidateInventoryBytes added in v0.7.0 

func ValidateInventoryBytes(byts []byte) (*StoredInventory, *Validation)

ValidateInventoryBytes parses and fully validates the byts as contents of an inventory.json file. This is mostly used for testing.

func ValidateNamaste added in v0.1.0 

func ValidateNamaste(ctx context.Context, fsys ocflfs.FS, name string) (err error)

ValidateNamaste validates a namaste declaration

func WriteDeclaration 

func WriteDeclaration(ctx context.Context, root ocflfs.FS, dir string, d Namaste) error

func WriteSpecFile 

func WriteSpecFile(ctx context.Context, fsys ocflfs.WriteFS, dir string, n Spec) (string, error)

Types

type ContentSource added in v0.0.23 

type ContentSource interface {
	// GetContent returns an FS and path to a file in FS for a file with the given digest.
	// If no content is associated with the digest, fsys is nil and path is an empty string.
	GetContent(digest string) (fsys fs.FS, path string)
}

ContentSource is used to access content with a given digest when creating and upadting objects.

type DigestMap 

type DigestMap map[string][]string

DigestMap maps digests to file paths.

func (DigestMap) AllPaths added in v0.8.0 

func (m DigestMap) AllPaths() []string

AllPaths returns a sorted slice of all path names in the DigestMap.

func (DigestMap) Clone added in v0.0.23 

func (m DigestMap) Clone() DigestMap

Clone returns a copy of dm

func (DigestMap) DigestFor added in v0.8.0 

func (m DigestMap) DigestFor(p string) string

DigestFor returns the digest for path p or an empty string if the digest is not present.

func (DigestMap) Eq 

func (m DigestMap) Eq(other DigestMap) bool

Eq returns true if m and the other have the same content: they have the same (normalized) digests corresponding to the same set of paths. If either map has a digest conflict (same digest appears twice with different case), Eq returns false.

func (DigestMap) Merge 

func (m1 DigestMap) Merge(m2 DigestMap, replace bool) (DigestMap, error)

Merge returns a new DigestMap constructed by normalizing and merging m1 and m2. If a paths has different digests in m1 and m2, an error returned unless replace is true, in which case the value from m2 is used.

func (DigestMap) Mutate added in v0.8.0 

func (m DigestMap) Mutate(fns ...PathMutation)

Mutate applies each path mutation function to paths for each digest in m. If the mutations remove all paths for the digest, the digest key is deleted from m. Mutate may make m invalid.

func (DigestMap) Normalize 

func (m DigestMap) Normalize() (norm DigestMap, err error)

Normalize checks if m is valid and returns a normalized copy (with lowercase digests) and sorted paths.

func (DigestMap) NumPaths added in v0.0.23 

func (m DigestMap) NumPaths() int

NumPaths returns the number of paths in m

func (DigestMap) PathMap 

func (m DigestMap) PathMap() PathMap

PathMap returns a PathMap with m's paths and corresponding digests. The returned PathMap may be invalid.

func (DigestMap) Paths 

func (m DigestMap) Paths() iter.Seq2[string, string]

Paths is an iterator that yields path/digest pairs in m. The order paths are yielded is not defined.

func (DigestMap) Valid 

func (m DigestMap) Valid() error

Valid returns a non-nil error if m is invalid.

type FixitySource added in v0.0.23 

type FixitySource interface {
	// GetFixity returns a DigestSet with alternate digests for the content with
	// the digest derived using the stage's primary digest algorithm.
	GetFixity(digest string) digest.Set
}

FixitySource is used to access alternate digests for content with a given digest (sha512 or sha256) when creating or updating objects.

type Inventory added in v0.7.0 

type Inventory struct {
	ID               string                     `json:"id"`
	Type             InventoryType              `json:"type"`
	DigestAlgorithm  string                     `json:"digestAlgorithm"`
	Head             VNum                       `json:"head"`
	ContentDirectory string                     `json:"contentDirectory,omitempty"`
	Manifest         DigestMap                  `json:"manifest"`
	Versions         map[VNum]*InventoryVersion `json:"versions"`
	Fixity           map[string]DigestMap       `json:"fixity,omitempty"`
}

Inventory represents the contents of an object's inventory.json file

func (Inventory) GetFixity added in v0.9.0 

func (inv Inventory) GetFixity(dig string) digest.Set

GetFixity provides alternate digests associated with all files with the given primary digest.

func (*Inventory) Validate added in v0.9.0 

func (inv *Inventory) Validate() *Validation

Validate validates inv using its declarate OCFL spec.

type InventoryBuilder added in v0.9.0 

type InventoryBuilder struct {
	// contains filtered or unexported fields
}

func NewInventoryBuilder added in v0.9.0 

func NewInventoryBuilder(prev *Inventory) *InventoryBuilder

Create a new inventory builder. If prev is not nil, the builder's initial Spec, ID, and ContentDirectory are used.

func (*InventoryBuilder) AddVersion added in v0.9.0 

func (b *InventoryBuilder) AddVersion(state DigestMap, alg digest.Algorithm, created time.Time, message string, user *User) *InventoryBuilder

AddVersion increments the inventory's head and adds a new version with the given stage, creation timestamp, message, and user information.

func (*InventoryBuilder) ContentDirectory added in v0.9.0 

func (b *InventoryBuilder) ContentDirectory(name string) *InventoryBuilder

Sets the inventory's content directory. Ignored if the builder was initialized with an existing inventory.

func (*InventoryBuilder) ContentPathFunc added in v0.9.0 

func (b *InventoryBuilder) ContentPathFunc(mutate PathMutation) *InventoryBuilder

ContentPathFunc sets a function used to generate content directory names for new manifest entries.

func (*InventoryBuilder) Finalize added in v0.9.0 

func (b *InventoryBuilder) Finalize() (*Inventory, error)

Finalize builds and validates a new inventory.

func (*InventoryBuilder) FixitySource added in v0.9.0 

func (b *InventoryBuilder) FixitySource(source FixitySource) *InventoryBuilder

func (*InventoryBuilder) ID added in v0.9.0 

func (b *InventoryBuilder) ID(id string) *InventoryBuilder

ID sets the inventory's ID

func (*InventoryBuilder) Padding added in v0.9.0 

func (b *InventoryBuilder) Padding(p int) *InventoryBuilder

Padding sets the inventory's version number padding. Ignored if the builder was initialized with an existing inventory.

func (*InventoryBuilder) Spec added in v0.9.0 

func (b *InventoryBuilder) Spec(spec Spec) *InventoryBuilder

Set the inventory's OCFL spec. Ignored if spec is a zero value.

type InventoryType added in v0.7.0 

type InventoryType struct {
	Spec
}

InventoryType represents an inventory type string for example: https://ocfl.io/1.0/spec/#inventory

func (InventoryType) MarshalText added in v0.7.0 

func (invT InventoryType) MarshalText() ([]byte, error)

func (InventoryType) String added in v0.7.0 

func (inv InventoryType) String() string

func (*InventoryType) UnmarshalText added in v0.7.0 

func (invT *InventoryType) UnmarshalText(t []byte) error

type InventoryVersion added in v0.9.0 

type InventoryVersion struct {
	Created time.Time `json:"created"`
	State   DigestMap `json:"state"`
	Message string    `json:"message,omitempty"`
	User    *User     `json:"user,omitempty"`
}

Version represents object version state and metadata

type MapDigestConflictErr 

type MapDigestConflictErr struct {
	Digest string
}

MapDigestConflictErr indicates same digest found multiple times in the digest map (i.e., with different cases)

func (*MapDigestConflictErr) Error 

func (d *MapDigestConflictErr) Error() string

type MapPathConflictErr 

type MapPathConflictErr struct {
	Path string
}

MapPathConflictErr indicates a path appears more than once in the digest map. It's also used in cases where the path as used as a directory in one instance and a file in another.

func (*MapPathConflictErr) Error 

func (p *MapPathConflictErr) Error() string

type MapPathInvalidErr 

type MapPathInvalidErr struct {
	Path string
}

MapPathInvalidErr indicates an invalid path in a Map.

func (*MapPathInvalidErr) Error 

func (p *MapPathInvalidErr) Error() string

type Namaste added in v0.0.24 

type Namaste struct {
	Type    string
	Version Spec
}

Namaste represents a NAMASTE declaration

func FindNamaste added in v0.0.24 

func FindNamaste(items []fs.DirEntry) (Namaste, error)

FindNamaste returns the NAMASTE declaration from a fs.DirEntry slice. An error is returned if the number of declarations is not one.

func ParseNamaste added in v0.0.24 

func ParseNamaste(name string) (n Namaste, err error)

func (Namaste) Body added in v0.0.24 

func (n Namaste) Body() string

Body returns the expected file contents of the namaste declaration

func (Namaste) IsObject added in v0.5.0 

func (n Namaste) IsObject() bool

IsObject returs true if n's type is 'ocfl_object'

func (Namaste) IsRoot added in v0.8.0 

func (n Namaste) IsRoot() bool

IsRoot returns true if n's type is 'ocfl'

func (Namaste) Name added in v0.0.24 

func (n Namaste) Name() string

Name returns the filename for n ('0=TYPE_VERSION') or an empty string if n is empty

type Object added in v0.1.0 

type Object struct {
	// contains filtered or unexported fields
}

Object represents and OCFL Object, typically part of a Root.

func NewObject added in v0.1.0 

func NewObject(ctx context.Context, fsys ocflfs.FS, dir string, opts ...ObjectOption) (*Object, error)

NewObject returns an *Object for managing the OCFL object at directory dir in fsys. The object doesn't need to exist when NewObject is called.

func (*Object) ApplyUpdatePlan added in v0.9.0 

func (obj *Object) ApplyUpdatePlan(ctx context.Context, update *UpdatePlan, src ContentSource) error

ApplyUpdatePlan applies an *UpdatePlan, resulting in a new object version. The *UpdatePlan should be created with Object.NewUpdatePlan. The internal state for obj is updated to reflect the new object inventory.

func (Object) ContentDirectory added in v0.9.0 

func (obj Object) ContentDirectory() string

ContentDirectory return "content" or the value set in the root inventory.

func (Object) DigestAlgorithm added in v0.9.0 

func (obj Object) DigestAlgorithm() digest.Algorithm

DigestAlgorithm returns sha512 unless sha256 is set in the root inventory.

func (Object) Exists added in v0.1.0 

func (obj Object) Exists() bool

Exists returns true if the object has an existing version.

func (Object) ExtensionNames added in v0.1.0 

func (obj Object) ExtensionNames(ctx context.Context) ([]string, error)

ExtensionNames returns the names of directories in the object's extensions directory.

func (Object) FS added in v0.1.0 

func (obj Object) FS() ocflfs.FS

FS returns the FS where object is stored.

func (Object) FixityAlgorithms added in v0.9.0 

func (obj Object) FixityAlgorithms() []string

FixityAlgorithms returns a slice of the keys from the `fixity` block of obj's return inventory. If obj does not have an inventory (i.e., because one has not been created yet), it returns nil.

func (Object) GetContent added in v0.9.0 

func (obj Object) GetContent(dig string) (ocflfs.FS, string)

GetContent implements ContentSource for Object, for use in a Stage.

func (Object) GetFixity added in v0.9.0 

func (obj Object) GetFixity(dig string) digest.Set

GetFixity implements the FixitySource interface for Object, for use in a Stage.

func (Object) Head added in v0.9.0 

func (obj Object) Head() VNum

Head returns the most recent version number. If obj has no root inventory, it returns the zero value.

func (Object) ID added in v0.7.1 

func (obj Object) ID() string

ID returns obj's inventory ID if the obj exists (its inventory is not nil). If obj does not exist but was constructed with Root.NewObject or with the ObjectWithID option, the ID given as an argument is returned. Otherwise, it returns an empty string.

func (*Object) InventoryBuilder added in v0.9.0 

func (obj *Object) InventoryBuilder() *InventoryBuilder

InventoryBuilder returns an *InventoryBuilder that can be used to generate new inventory's for the object.

func (Object) InventoryDigest added in v0.9.0 

func (obj Object) InventoryDigest() string

InventoryDigest returns the digest of the object's root inventory using the declarate digest algorithm. It is the expected content of the root inventory's sidecar file.

func (Object) Manifest added in v0.9.0 

func (obj Object) Manifest() DigestMap

Manifest returns a copy of the root inventory manifest. If the object has no root inventory (e.g., it doesn't yet exist), nil is returned.

func (*Object) NewUpdatePlan added in v0.9.0 

func (obj *Object) NewUpdatePlan(stage *Stage, msg string, user User, opts ...ObjectUpdateOption) (*UpdatePlan, error)

NewUpdatePlan builds a new object inventory using stage's state and returns an *UpdatePlan that can be used to apply the changes. It does not apply the update plan.

func (Object) Path added in v0.1.0 

func (obj Object) Path() string

Path returns the Object's path relative to its FS.

func (Object) ReadOnly added in v0.9.1 

func (obj Object) ReadOnly() error

ReadOnly returns an error if obj does not support updates. Updates may be prohibited if obj's storage backend does not support writes or if it was initialized using an explicit inventory (using ObjectWithInventory) and without root sidecar validation (using ObjectSkipRootSidecarValidation).

func (Object) Root added in v0.7.0 

func (o Object) Root() *Root

Root returns the object's Root, if known. It is nil unless the *Object was created using Root.NewObject

func (Object) Spec added in v0.9.0 

func (o Object) Spec() Spec

Spec returns the OCFL spec number from the object's root inventory, or an empty Spec if the root inventory does not exist.

func (*Object) Update added in v0.9.0 

func (obj *Object) Update(ctx context.Context, stage *Stage, msg string, user User, opts ...ObjectUpdateOption) (*UpdatePlan, error)

Update creates a new object version with stage's state, the given version message, and the user. To create the new object version, an *UpdatePlan is created with Object.NewUpdatePlan and applied with Object.ApplyUpdatePlan. The *UpdatePlan is returned even if an error occurs while applying the plan. The *UpdatePlan can be be retried, by calling Object.ApplyUpdatePlan again, or reverted, by calling UpdatePlan.Revert.

func (Object) Version added in v0.9.0 

func (obj Object) Version(v int) *ObjectVersion

Version returns an *ObjectVersion that can be used to access details for the version with the given number (1...HEAD) from the root inventory. For example, v == 1 refers to "v1" or "v001" version block. If v < 1, the most recent version is returned. If the version does not exist, nil is returned.

func (*Object) VersionFS added in v0.9.0 

func (obj *Object) VersionFS(ctx context.Context, v int) (fs.FS, error)

VersionFS returns an io/fs.FS representing the logical state for the version with the given number (1...HEAD). If v < 1, the most recent version is used.

func (*Object) VersionStage added in v0.9.0 

func (obj *Object) VersionStage(v int) *Stage

VersionStage returns a *Stage matching the content of the version with the given number (1...HEAD). If ther version does not exist, nil is returned.

type ObjectOption added in v0.1.0 

type ObjectOption func(*newObjectConfig)

ObjectOptions are used to configure the behavior of NewObject()

func ObjectMustExist added in v0.5.0 

func ObjectMustExist() ObjectOption

ObjectMustExists is an ObjectOption used to indicate that the initialized object instance must be an existing OCFL object.

func ObjectSkipRootSidecarValidation added in v0.9.1 

func ObjectSkipRootSidecarValidation() ObjectOption

ObjectSkipRootSidecarValidation is used to skip validating the inventory.json digest with the root inventory sidecar file during initialization.

func ObjectWithID added in v0.9.0 

func ObjectWithID(id string) ObjectOption

ObjectWithID is an ObjectOption used to set an explict object ID in contexts where the object ID is not known or must match a certain value.

func ObjectWithInventory added in v0.9.1 

func ObjectWithInventory(inv *StoredInventory) ObjectOption

ObjectWithInventory is used to initialize an *Object using an existing *StoredInventory value. It can be used with an inventory cache to minimize requests to the object's storage backed. Unless it is combined with the ObjectSkipRootSidecarValidation option, inv's digest will be validated against the root inventory sidecar file.

type ObjectState 

type ObjectState struct {
	Spec        Spec            // the OCFL spec from the object's NAMASTE declaration file
	VersionDirs VNums           // version directories found in the object directory
	SidecarAlg  string          // digest algorithm used by the inventory sidecar file
	Invalid     []string        // non-conforming directory entries in the object root (max of 8)
	Flags       objectStateFlag // boolean attributes of the object root
}

ObjectState provides details of an OCFL object root based on the names of files and directories in the object's root. ParseObjectDir is typically used to create a new ObjectState from a slice of fs.DirEntry values.

func ParseObjectDir added in v0.1.0 

func ParseObjectDir(entries []fs.DirEntry) *ObjectState

ParseObjectDir returns a new ObjectState based on contents of an object root directory.

func (ObjectState) Empty added in v0.1.0 

func (state ObjectState) Empty() bool

Empty returns true if the object root directory is empty

func (ObjectState) HasExtensions added in v0.1.0 

func (state ObjectState) HasExtensions() bool

HasExtensions returns true if state's HasExtensions flag is set

func (ObjectState) HasInventory added in v0.1.0 

func (state ObjectState) HasInventory() bool

HasInventory returns true if state's HasInventory flag is set

func (ObjectState) HasLogs added in v0.3.1 

func (state ObjectState) HasLogs() bool

HasLogs returns true if state's HasLogs flag is set

func (ObjectState) HasNamaste added in v0.1.0 

func (state ObjectState) HasNamaste() bool

HasNamaste returns true if state's HasNamaste flag is set

func (ObjectState) HasSidecar added in v0.1.0 

func (state ObjectState) HasSidecar() bool

HasSidecar returns true if state's HasSidecar flag is set

func (ObjectState) HasVersionDir added in v0.1.0 

func (state ObjectState) HasVersionDir(v VNum) bool

HasVersionDir returns true if the state's VersionDirs includes v

func (ObjectState) Namaste added in v0.6.0 

func (state ObjectState) Namaste() Namaste

Namaste returns state's Namaste value, which may be a zero value.

type ObjectUpdateOption added in v0.9.0 

type ObjectUpdateOption func(*objectUpdateOptions)

ObjectUpdateOptions are optional function arguments used to configure new an Object's UpdatePlan.

func UpdateWithContentPathFunc added in v0.9.0 

func UpdateWithContentPathFunc(mutate PathMutation) ObjectUpdateOption

UpdateWithContentPathFunc is used to set a function for enforcing naming conventions for content paths in the new inventory.

func UpdateWithGoLimit added in v0.9.0 

func UpdateWithGoLimit(gos int) ObjectUpdateOption

UpdateWithGoLimit sets the number of goroutines used to run concurrent steps when running the UpdatePlan.

func UpdateWithLogger added in v0.9.0 

func UpdateWithLogger(logger *slog.Logger) ObjectUpdateOption

UpdateWithLogger sets the logger used to log message from each step of the UpdatePlan.

func UpdateWithNewHead added in v0.9.0 

func UpdateWithNewHead(v int) ObjectUpdateOption

UpdateWithNewHead is used to enforce the expected version number (without padding) for the version created with the update. Without this, the new version increments the existing version number, whatever it may be.

func UpdateWithOCFLSpec added in v0.9.0 

func UpdateWithOCFLSpec(s Spec) ObjectUpdateOption

UpdateWithOCFLSpec sets the OCFL specification for the new object version

func UpdateWithUnchangedVersionState added in v0.9.0 

func UpdateWithUnchangedVersionState() ObjectUpdateOption

UpdateWithUnchangedVersionState is used to allow updates that don't change the version state. Without this, updates with the same state will result in an error.

func UpdateWithVersionCreated added in v0.9.0 

func UpdateWithVersionCreated(t time.Time) ObjectUpdateOption

UpdateWithVersionCreated sets the 'created' timestamp for the object version created with the update.

type ObjectValidation added in v0.1.0 

type ObjectValidation struct {
	Validation
	// contains filtered or unexported fields
}

ObjectValidation is used to configure and track results from an object validation process.

func ValidateObject added in v0.3.0 

func ValidateObject(ctx context.Context, fsys ocflfs.FS, dir string, opts ...ObjectValidationOption) *ObjectValidation

ValidateObject fully validates the OCFL Object at dir in fsys

func (*ObjectValidation) Add added in v0.1.0 

func (v *ObjectValidation) Add(v2 *Validation)

Add adds and logs all fatal errors and warning from the validation

func (*ObjectValidation) AddFatal added in v0.1.0 

func (v *ObjectValidation) AddFatal(errs ...error)

AddFatal adds fatal errors to the validation

func (*ObjectValidation) AddWarn added in v0.1.0 

func (v *ObjectValidation) AddWarn(errs ...error)

AddWarn adds warning errors to the object validation and logs the errors using the object validations logger, if set.

func (*ObjectValidation) DigestConcurrency added in v0.4.0 

func (v *ObjectValidation) DigestConcurrency() int

DigestConcurrency returns the configured number of go routines used to read and digest contents during validation. The default value is runtime.NumCPU().

func (*ObjectValidation) Logger added in v0.1.0 

func (v *ObjectValidation) Logger() *slog.Logger

Logger returns the validation's logger, which is nil by default.

func (*ObjectValidation) PrefixAdd added in v0.1.0 

func (v *ObjectValidation) PrefixAdd(prefix string, v2 *Validation)

PrefixAdd adds and logs all fatal errors and warning from the validation, prepending each error with the prefix.

func (*ObjectValidation) SkipDigests added in v0.1.0 

func (v *ObjectValidation) SkipDigests() bool

SkipDigests returns true if the validation is configured to skip digest checks. It is false by default.

func (*ObjectValidation) ValidationAlgorithms added in v0.4.0 

func (v *ObjectValidation) ValidationAlgorithms() digest.AlgorithmRegistry

ValidationAlgorithms returns the registry of digest algorithms the object validation is configured to use. The default value is digest.DefaultRegistry

type ObjectValidationOption added in v0.1.0 

type ObjectValidationOption func(*ObjectValidation)

func ValidationAlgorithms added in v0.4.0 

func ValidationAlgorithms(reg digest.AlgorithmRegistry) ObjectValidationOption

ValidationAlgorithms sets registry of available digest algorithms for fixity validation.

func ValidationDigestConcurrency added in v0.4.0 

func ValidationDigestConcurrency(num int) ObjectValidationOption

ValidationDigestConcurrency is used to set the number of go routines used to read and digest contents during validation.

func ValidationLogger added in v0.1.0 

func ValidationLogger(logger *slog.Logger) ObjectValidationOption

ValidationLogger sets the *slog.Logger that should be used for logging validation errors and warnings.

func ValidationSkipDigest added in v0.1.0 

func ValidationSkipDigest() ObjectValidationOption

type ObjectVersion added in v0.1.0 

type ObjectVersion struct {
	// contains filtered or unexported fields
}

ObjectVersion is used to access version information from an object's root inventory.

func (ObjectVersion) Created added in v0.1.0 

func (o ObjectVersion) Created() time.Time

Created returns the version's created timestamp

func (ObjectVersion) Message added in v0.1.0 

func (o ObjectVersion) Message() string

Message returns the version's message

func (ObjectVersion) State added in v0.1.0 

func (o ObjectVersion) State() DigestMap

State returns a copy of the version's state

func (ObjectVersion) User added in v0.1.0 

func (o ObjectVersion) User() *User

User returns the version's user information, which may be nil

func (ObjectVersion) VNum added in v0.9.0 

func (o ObjectVersion) VNum() VNum

VNum returns o's version number

type PathMap added in v0.0.23 

type PathMap map[string]string

PathMap maps filenames to digest strings.

func (PathMap) DigestMap added in v0.0.23 

func (pm PathMap) DigestMap() DigestMap

DigestMap returns a new DigestMap using the pathnames and digests in pm. The resulting DigestMap may be invalid if pm includes invalid paths or digests.

func (PathMap) SortedPaths added in v0.8.0 

func (pm PathMap) SortedPaths() iter.Seq2[string, string]

SortedPaths is an iterator that yields the path/digest pairs in pm in sorted order (by pathname).

type PathMutation added in v0.8.0 

type PathMutation func(oldPaths []string) (newPaths []string)

PathMutation is used with DigestMap.Mutate to change paths names in a DigestMap

func RemovePath added in v0.8.0 

func RemovePath(name string) PathMutation

RemovePath returns a PathMutation that removes name

func RenamePaths added in v0.8.0 

func RenamePaths(src, dst string) PathMutation

RenamePaths returns a PathMutation function that renames occurences of src to dst. If src matches a full path, it is replaced with dst. If src matches a directory (including '.'), all occurences of the directory prefix are replaced with dst (which may be '.').

type PlanStep added in v0.9.0 

type PlanStep struct {
	// contains filtered or unexported fields
}

UndoStep is a single step in an UpdatePlan.

func (PlanStep) Completed added in v0.9.0 

func (step PlanStep) Completed() bool

Completed returns true if the step ran without error or was successfully reverted.

func (PlanStep) ContentDigest added in v0.9.0 

func (step PlanStep) ContentDigest() string

ContentDigest returns the digest of the content (if any) copied during the step.

func (PlanStep) ErrMsg added in v0.9.0 

func (step PlanStep) ErrMsg() string

ErrMsg returns any error message from the step's last Run.

func (PlanStep) MarshalBinary added in v0.9.0 

func (step PlanStep) MarshalBinary() ([]byte, error)

func (PlanStep) Name added in v0.9.0 

func (step PlanStep) Name() string

Name returns the step's unique name

func (*PlanStep) Revert added in v0.9.0 

func (step *PlanStep) Revert(ctx context.Context, objFS ocflfs.FS, objDir string, src ContentSource) error

Revert calls step's undo function if the step is marked as complete. If the undo function returns an error, the error message is saved as RevertErr. If Revert does not result in an error, the step is marked as incomplete and UndoErr is cleared.

func (*PlanStep) Run added in v0.9.0 

func (step *PlanStep) Run(ctx context.Context, objFS ocflfs.FS, objDir string, src ContentSource) error

Run runs the step's function if the step is not marked as complete, recording any error message to Err. If the step does not return an error, it is marked as complete and any previous error message is cleared.

func (*PlanStep) Size added in v0.9.0 

func (step *PlanStep) Size() int64

Size returns the number of bytes copied to the object as part of step's run action. This value is available after step has run.

func (*PlanStep) UnmarshalBinary added in v0.9.0 

func (step *PlanStep) UnmarshalBinary(b []byte) error

type PlanSteps added in v0.9.0 

type PlanSteps []PlanStep

PlanSteps is a series of named steps for performing an object update and rolling it back if necessary.

func (PlanSteps) Eq added in v0.9.0 

func (s PlanSteps) Eq(s2 PlanSteps) bool

type Root added in v0.1.0 

type Root struct {
	// contains filtered or unexported fields
}

Root represents an OCFL Storage Root.

func NewRoot added in v0.1.0 

func NewRoot(ctx context.Context, fsys ocflfs.FS, dir string, opts ...RootOption) (*Root, error)

NewRoot returns a new *Root for working with the OCFL storage root at directory dir in fsys. It can be used to initialize new storage roots if the InitRoot option is used, fsys is an ocfl.WriteFS, and dir is a non-existing or empty directory.

func (*Root) Description added in v0.1.0 

func (r *Root) Description() string

Description returns the description string from the storage roots `ocfl_layout.json` file, which may be empty.

func (*Root) FS added in v0.1.0 

func (r *Root) FS() ocflfs.FS

FS returns the Root's FS

func (*Root) Layout added in v0.1.0 

func (r *Root) Layout() extension.Layout

Layout returns the storage root's layout, which may be nil.

func (*Root) LayoutName added in v0.1.0 

func (r *Root) LayoutName() string

LayoutName returns the name of the root's layout extension or an empty string if the root has no layout.

func (*Root) NewObject added in v0.1.0 

func (r *Root) NewObject(ctx context.Context, id string, opts ...ObjectOption) (*Object, error)

NewObject returns an *Object for managing the OCFL object with the given ID in the root. If the object does not exist, the returned *Object can be used to create it. If the Root has not storage layout for resovling object IDs, the returned error is ErrLayoutUndefined.

func (*Root) NewObjectDir added in v0.3.0 

func (r *Root) NewObjectDir(ctx context.Context, dir string, opts ...ObjectOption) (*Object, error)

NewObjectDir returns an *Object for managing the OCFL object at path dir in root. If the object does not exist, the returned *Object can be used to create it.

func (*Root) ObjectDeclarations added in v0.6.0 

func (r *Root) ObjectDeclarations(ctx context.Context) iter.Seq2[*ocflfs.FileRef, error]

ObjectDeclarations returns an iterator that yields all OCFL object declaration files in r. If an error occurs during iteration, it is returned by the error function.

func (*Root) Objects added in v0.1.0 

func (r *Root) Objects(ctx context.Context, opts ...ObjectOption) iter.Seq2[*Object, error]

Objects returns an iterator that yields objects or an error for every object declaration file in the root. Objects are yielded in arbitrary order.

func (*Root) ObjectsBatch added in v0.6.0 

func (r *Root) ObjectsBatch(ctx context.Context, numgos int, opts ...ObjectOption) iter.Seq2[*Object, error]

ObjectsBatch returns an iterator that yields objects or an error for every object declaration file in the root. Objects are read in numgos go routines and are yielded in arbitrary order.

func (*Root) Path added in v0.1.0 

func (r *Root) Path() string

Path returns the root's dir relative to its FS

func (*Root) ResolveID added in v0.3.0 

func (r *Root) ResolveID(id string) (string, error)

ResolveID resolves the object id to a path relative to the root. If the root has no layout, the returned error is ErrLayoutUndefined.

func (*Root) Spec added in v0.1.0 

func (r *Root) Spec() Spec

Spec returns the root's OCFL specification number

func (*Root) ValidateObject added in v0.3.0 

func (r *Root) ValidateObject(ctx context.Context, id string, opts ...ObjectValidationOption) *ObjectValidation

ValidateObject validates the object with the given id. If the id cannot be resolved, the error is reported as a fatal error in the returned *ObjectValidation.

func (*Root) ValidateObjectDir added in v0.3.0 

func (r *Root) ValidateObjectDir(ctx context.Context, dir string, opts ...ObjectValidationOption) *ObjectValidation

ValidateObjectDir validates the object at a path relative to the root.

type RootOption added in v0.1.0 

type RootOption func(*Root)

RootOption is used to configure the behavior of NewRoot()

func InitRoot added in v0.1.0 

func InitRoot(spec Spec, layoutDesc string, extensions ...extension.Extension) RootOption

InitRoot returns a RootOption for initializing a new storage root as part of the call to NewRoot().

type Spec 

type Spec string

Spec represent an OCFL specification number

func (Spec) Cmp 

func (v1 Spec) Cmp(v2 Spec) int

Cmp compares Spec v1 to another v2. - If v1 is less than v2, returns -1. - If v1 is the same as v2, returns 0 - If v1 is greater than v2, returns 1 - any valid spec is greater than an invalid spec. - if both specs are invlid, Cmp panics.

func (Spec) Empty 

func (s Spec) Empty() bool

func (Spec) InventoryType added in v0.7.0 

func (s Spec) InventoryType() InventoryType

InventoryType returns n as an InventoryType

func (Spec) Valid added in v0.0.24 

func (s Spec) Valid() error

type Stage 

type Stage struct {
	// State is a DigestMap representing the new object version state.
	State DigestMap
	// DigestAlgorithm is the primary digest algorithm (sha512 or sha256) used by the stage
	// state.
	DigestAlgorithm digest.Algorithm
	// ContentSource is used to access new content needed to construct
	// an object. It may be nil
	ContentSource
	// FixitySource is used to access fixity information for new
	// content. It may be nil
	FixitySource
}

Stage is used to create/update objects.

func StageBytes added in v0.0.23 

func StageBytes(content map[string][]byte, alg digest.Algorithm, fixity ...digest.Algorithm) (*Stage, error)

StageBytes builds a stage from a map of filenames to file contents

func StageDir added in v0.0.23 

func StageDir(ctx context.Context, fsys fs.FS, dir string, alg digest.Algorithm, fixity ...digest.Algorithm) (*Stage, error)

StageDir builds a stage based on the contents of the directory dir in FS. Files in dir and its subdirectories are digested with the given digest algorithms and added to the stage. Hidden files are ignored. The alg argument must be sha512 or sha256.

func StageFiles added in v0.8.0 

func StageFiles(ctx context.Context, files iter.Seq[*fs.FileRef], alg digest.Algorithm, fixity ...digest.Algorithm) (*Stage, error)

StageFiles buils a stage from entries in files. Files are digested with the given digest algorithms and added to the stage. The alg argument must be sha512 or sha256.

func (Stage) HasContent added in v0.0.23 

func (s Stage) HasContent(digest string) bool

HasContent returns true if the stage's content source provides an FS and path for the digest

func (*Stage) Overlay added in v0.0.23 

func (s *Stage) Overlay(stages ...*Stage) error

Overlay merges the state and content/fixity sources from all stages into s. All stages mush share the same digest algorithm.

type StoredInventory added in v0.9.0 

type StoredInventory struct {
	Inventory
	// contains filtered or unexported fields
}

StoredInventory is an unmarshaled inventory with the digest of its source data.

func NewStoredInventory added in v0.9.1 

func NewStoredInventory(r io.Reader) (*StoredInventory, error)

NewStoredInventory reads all bytes from r and parses the contents as as Inventory. The returned *StoredInventory is valid and includes the a digest of the bytes read from r.

func ReadInventory added in v0.1.0 

func ReadInventory(ctx context.Context, fsys ocflfs.FS, dir string) (*StoredInventory, error)

ReadInventory reads the inventory.json file in dir and validates it. It returns an error if the inventory can't be parsed or if it is invalid.

func (StoredInventory) Digest added in v0.9.0 

func (inv StoredInventory) Digest() string

Digest returns the digest of inventory.json file used to create inv using the digest algorithm specified in the inventory.

func (StoredInventory) MarshalBinary added in v0.9.1 

func (inv StoredInventory) MarshalBinary() ([]byte, error)

MarshalBinary implements encoding.BinaryMarshaler for StoredInventory. It returns the full contents of the inventory.json file used to create inv.

func (*StoredInventory) UnmarshalBinary added in v0.9.1 

func (inv *StoredInventory) UnmarshalBinary(data []byte) error

UnmarshalBinary implements encoding.BinaryUnmarshaler for *StoredInventory. The data argument must be the full contents of an inventory.json, otherwise inv's Digest value may not match its sidecar. (BinaryUnmarshaler is used instead of json.Unmarshaler because the latter may not preserve the exact bytestream, for example leading or trailing whitespace.)

func (StoredInventory) ValidateSidecar added in v0.9.0 

func (inv StoredInventory) ValidateSidecar(ctx context.Context, fsys ocflfs.FS, dir string) error

ValidateSidecar reads the inventory sidecar with inv's digest algorithm (e.g., inventory.json.sha512) in directory dir and return an error if the sidecar content is not formatted correctly or if the inv's digest doesn't match the value found in the sidecar.

type UpdatePlan added in v0.9.0 

type UpdatePlan struct {
	// contains filtered or unexported fields
}

UpdatePlan is a sequence of steps (PlanStep) for updating an OCFL object. It allows updates to be interrupted, resumed, retried or reverted. To update an object, each PlanStep in the UpdatePlan must run to completion. The result of each step (whether the step completed successfully or resulted in an error) is stored so that repeated updates resume where the previous run stopped or failed. Each PlanStep includes compensating actions for reverting partial updates that cannot be completed.

func (*UpdatePlan) Apply added in v0.9.0 

func (u *UpdatePlan) Apply(ctx context.Context, objFS ocflfs.FS, objDir string, src ContentSource) (*StoredInventory, error)

Apply runs incomplete steps in the UpdatePlan and returns the object's new *StoredInventory if the updated succeeded. If any step in the UpdatePlan results in an error, execution stops and the error is returned. Some steps in the plan may run concurrently. Use SetGoLimit to set number of goroutines used to run concurrent steps.

func (UpdatePlan) BaseInventoryDigest added in v0.9.0 

func (u UpdatePlan) BaseInventoryDigest() string

BaseInventoryDigest returns the digest of the object's existing inventory.json. It returns an empty string if the UpdatePlan would create a new object.

func (UpdatePlan) Completed added in v0.9.0 

func (u UpdatePlan) Completed() bool

Completed return true if all u's steps are marked as completed

func (UpdatePlan) CompletedSteps added in v0.9.0 

func (u UpdatePlan) CompletedSteps() iter.Seq[*PlanStep]

CompletedSteps is an iterator over completed steps in reverse order (most recently completed first).

func (UpdatePlan) Eq added in v0.9.0 

func (u UpdatePlan) Eq(other *UpdatePlan) bool

Eq returns true if u and other represent that same update plan, with the same set of steps.

func (UpdatePlan) Err added in v0.9.0 

func (u UpdatePlan) Err() error

Err returns an error that wraps all errors found in the u's steps.

func (*UpdatePlan) IncompleteSteps added in v0.9.0 

func (u *UpdatePlan) IncompleteSteps() iter.Seq[*PlanStep]

IncompleteSteps is an iterator over incomplete steps in u. Incomplete steps may have errors.

func (UpdatePlan) MarshalBinary added in v0.9.0 

func (u UpdatePlan) MarshalBinary() ([]byte, error)

MarshalBinary returns a binary representation of u

func (UpdatePlan) NextHead added in v0.9.1 

func (u UpdatePlan) NextHead() VNum

NextHead returns the number for the new object version to be created with the update.

func (UpdatePlan) NextInventoryDigest added in v0.9.1 

func (u UpdatePlan) NextInventoryDigest() string

NextInventoryDigest returns the digest of the inventory.json contents for the object version to be created with the update.

func (*UpdatePlan) ObjectID added in v0.9.0 

func (u *UpdatePlan) ObjectID() string

ObjectID returns the ID of the OCFL Object that the UpdatePlan must be applied to.

func (*UpdatePlan) Revert added in v0.9.0 

func (u *UpdatePlan) Revert(ctx context.Context, objFS ocflfs.FS, objDir string, src ContentSource) error

Revert calls the 'Revert' function on all Completed steps in u's update plan unless all steps have been completed. If all steps have been completed Revert has no effect and returns an ErrRevertUpdate.

func (UpdatePlan) Steps added in v0.9.0 

func (u UpdatePlan) Steps() iter.Seq[*PlanStep]

Steps iterates over all steps in the update plan

func (*UpdatePlan) UnmarshalBinary added in v0.9.0 

func (u *UpdatePlan) UnmarshalBinary(b []byte) error

UnmarshalBinary decodes b as a binary representation of an UpdatePlan and sets u to match.

type User 

type User struct {
	Name    string `json:"name"`
	Address string `json:"address,omitempty"`
}

User is a generic user information struct

type VNum 

type VNum struct {
	// contains filtered or unexported fields
}

VNum represents an OCFL object version number (e.g., "v1", "v02"). A VNum has a sequence number (1,2,3...) and a padding number, which defaults to zero. The padding is the maximum number of numeric digits the version number can include (a padding of 0 is no maximum). The padding value constrains the maximum valid sequence number.

func MustParseVNum 

func MustParseVNum(str string) VNum

MustParseVNum parses str as a VNUm and returns a new VNum. It panics if str cannot be parsed as a VNum.

func V 

func V(ns ...int) VNum

V returns a new Vnum. The first argument is a sequence number. An optional second argument can be used to set the padding. Additional arguments are ignored. Without any arguments, V() returns a zero value VNum.

func (VNum) First 

func (v VNum) First() bool

First returns true if v is a version 1.

func (VNum) IsZero 

func (v VNum) IsZero() bool

IsZero returns if v is the zero value

func (VNum) Lineage added in v0.1.0 

func (v VNum) Lineage() VNums

Lineage returns a VNums with v as the head.

func (VNum) MarshalText 

func (v VNum) MarshalText() ([]byte, error)

func (VNum) Next 

func (v VNum) Next() (VNum, error)

Next returns the next ocfl.VNum after v with the same padding. A non-nil error is returned if padding > 0 and next would overflow the padding

func (VNum) Num 

func (v VNum) Num() int

Num returns v's number as an int

func (VNum) Padding 

func (v VNum) Padding() int

Padding returns v's padding number.

func (VNum) String 

func (v VNum) String() string

String returns string representation of v

func (*VNum) UnmarshalText 

func (v *VNum) UnmarshalText(text []byte) error

func (VNum) Valid 

func (v VNum) Valid() error

Valid returns an error if v is invalid

type VNums 

type VNums []VNum

VNums is a slice of VNum elements

func (VNums) Head 

func (vs VNums) Head() VNum

Head returns the last VNum in vs.

func (VNums) Len 

func (vs VNums) Len() int

Len implements sort.Interface on VNums

func (VNums) Less 

func (vs VNums) Less(i, j int) bool

Less implements sort.Interface on VNums

func (VNums) Padding 

func (vs VNums) Padding() int

Padding returns the padding for the VNums in vs

func (VNums) Swap 

func (vs VNums) Swap(i, j int)

Swap implements sort.Interface on VNums

func (VNums) Valid 

func (vs VNums) Valid() error

Valid returns a non-nill error if VNums is empty, is not a continuous sequence (1,2,3...), has inconsistent padding or padding overflow.

type Validation added in v0.1.0 

type Validation struct {
	// contains filtered or unexported fields
}

Validation represents multiple fatal errors and warning errors.

func (*Validation) Add added in v0.1.0 

func (v *Validation) Add(v2 *Validation)

Add adds all fatal errors and warnings from another validation to v.

func (*Validation) AddFatal added in v0.1.0 

func (v *Validation) AddFatal(errs ...error)

AddFatal adds fatal errors to the validation

func (*Validation) AddWarn added in v0.1.0 

func (v *Validation) AddWarn(errs ...error)

AddWarn adds warning errors to the validation

func (*Validation) Err added in v0.1.0 

func (v *Validation) Err() error

Err returns an error wrapping all the validation's fatal errors, or nil if there are no fatal errors.

func (*Validation) Errors added in v0.1.0 

func (v *Validation) Errors() []error

Errors returns a slice of all the fatal errors.

func (*Validation) WarnErr added in v0.1.0 

func (v *Validation) WarnErr() error

WarnErr returns an error wrapping all the validation's warning errors, or nil if there are none.

func (*Validation) WarnErrors added in v0.1.0 

func (v *Validation) WarnErrors() []error

WarnErrors returns a slice of all the warning errors.

type ValidationError added in v0.1.0 

type ValidationError struct {
	validation.ValidationCode
	Err error
}

ValidationError is an error that includes a reference to a validation error code from the OCFL spec.

func (*ValidationError) Error added in v0.1.0 

func (ver *ValidationError) Error() string

func (*ValidationError) Unwrap added in v0.1.0 

func (ver *ValidationError) Unwrap() error

Source Files

View all Source files

Directories

Path Synopsis
examples
listobjects command
update command
validate command
fs
http
package http implements and http-based backend that supports basic object access.
 package http implements and http-based backend that supports basic object access.
local
s3
s3/example/allops command
s3/example/etag command
s3/example/large-copy command
s3/example/readir command
s3/internal/mock
internal
logical-fs
ocfltest
pipeline
testutil
code
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.