community-repo/working-groups/distributors

Distribution Working Group

• Intro for Leads

Tasks

• Bounty: Collect info for group reports
• Develop tools to implement Quality of Service, https://github.com/Joystream/community-repo/issues/656
• Deputy role: Based on available buckets, which channels should be distributed where?
• Create DP map: discussion

Image source: https://telemetry.joystream.org/#/Joystream

Olympia

Reports now on Notion

Giza

With release of the Giza Testnet selected channels were migrated from Sumer previous IPFS based storage system to complete in-house hosting with storage nodes for bigger portions of assets and local distribution nodes (read first).

Details about inception of the group are reported here (@freakstatic assigned this task to @l1dev who handed it @ilich).

• Lead: @l1dev (Concept)
• Contact: #distributors

Reports

• Giza 42

Tools

• GraphQL Playground comes with every Query node: https://hydra.joystream.org/graphql (see Queries below)

• Workers: https://joystreamstats.live/tokenomics (look for distributionWorkingGroup)

• Test availability of bags on all nodes: https://joystreamstats.live/distribution

Queries

Query nodes accept a specific syntax.

Each query takes optional offset and limit arguments (up to 10k depending on setup). When pasting a query he PRETTIFY button in graphql expands it. query { distributionBucketOperators (limit: 2, offset: 3) { workerId } }

This returns workers 4 and 7 (see sorting below). Note that features like advanced filtering are not in scope and need to be implemented client side.

For more details start a history lesson. Do not miss to try below queries.

Sorting queries

orderBy: [DistributionBucketOperatorOrderByInput!]
enum DistributionBucketOperatorOrderByInput {
createdAt_ASC
createdAt_DESC
updatedAt_ASC
updatedAt_DESC
deletedAt_ASC
deletedAt_DESC
distributionBucket_ASC
distributionBucket_DESC
workerId_ASC
workerId_DESC
status_ASC
status_DESC
metadata_ASC
metadata_DESC
}

Distribution Families with endpoint operators

query { distributionBucketFamilies { id metadata{description} buckets {id distributing acceptingNewBags operators {workerId metadata{extra nodeEndpoint nodeLocation{countryCode,coordinates{latitude longitude}}}} } } }

Distribution Buckets with bags

query distributionBuckets { distributionBuckets { id createdAt distributing acceptingNewBags operators { id workerId metadata {nodeEndpoint} } bags { id } }}

Find asset locations

Filters need to be applied to the first (or second) level (query node doesn't support nested queries:

query { storageDataObjects(where: { id_eq: "9506" }) {
    id storageBag {
      id
      distributionBuckets {
        id operators {
          metadata { nodeEndpoint } } } } } }

Storage Objects with license, bucket status and location

For curators to review new entries - the lead can assign each offset to a worker:

query getDataObjectsByVideoId { storageDataObjects(limit: 100, offset: 1) { id size deletionPrize storageBagId ipfsHash videomedia { id description licenseId license { code } media { storageBag { id owner { __typename } storageBuckets { id operatorMetadata { nodeLocation { countryCode } nodeEndpoint } createdById operatorStatus { __typename } dataObjectsSize acceptingNewBags dataObjectsCount dataObjectsSizeLimit } distributionBuckets { id bucketIndex familyId distributing createdById acceptingNewBags } } } } } }