community-repo/scripts

Here you find scripts and info how to retrieve data from the joystream chain.

There are several ways to access the chain storage. The easiest is the governance app pioneer.

• How to use the Chain State in Pioneer
• How to use the Javascript page in Pioneer
• joystream-cli
• api

Scripts

• joystream-api - examples for the joystream api
• Council minutes script - generates Council reports in Markdown
• sp-downloader-tester - measure storage provider speed
• node-setup-script - eases the installation of a validator node

Webapps

• validator-report-ui - frontend to show eras for selected validator
• validator-report-backend - stores validator data for each era

Telgram & Discord Bots

• joystream-leaderboard-bot - show FM stats + faucet
• joystream-scoring-period-bot - sends scoring period deadlines to Telegram
• joystreamtelegrambot - posts proposal updates and forum posts on Discord and Telegram
• joystreamvideobot - posts video uploads to #video on Discord
• storagesize-bot - posts storage provider space usage to #storage-provider on Discord

Contact

For usage questions: #tech-support.

To join development come to #operations and check Operations reports

Chain State

First select Fully Featured for interface operation mode on Settings and click Save.

Go to Chain State

To see the balance of an account navigate to system.account, select or enter a key and press the + button.

When you find a member ID and want to know, who it is, use members.membershipById.

If the handle is known: members.membershipById. The result is the membership ID to retrieve the membership like above. It expects hexadecimal bytes prefixed with 0x as input:

• Use a hex converter and prefix it with 0x.
• With node.js: Buffer.from("handle").toString('hex'))
• On JS enter console.log('handle'.split("").map(c => c.charCodeAt(0).toString(16).padStart(2, "0")).join("")) (save as string to hex for later by clicking on the disk button)

With the member info two keys are included, root_account and root_account. These are usually equal, except someone change the controller.

members.membershipById: Membership
{
  handle: joystreamstats,
  avatar_uri: https://joystreamstats.live/favicon.ico,
  about: https://joystreamstats.live,
  registered_at_block: 1,
  registered_at_time: 1,613,791,830,000,
  entry: {
    Genesis: null
  },
  suspended: false,
  subscription: null,
  root_account: 5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3,
  controller_account: 5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3
}

You can try this key with system.account to get the current balance.

For all above and generally it is possible to disable the input to retrieve a list of all entries (equivalent to for example api.query.system.account.entries() - see double maps).

JS

To use the Javascript page in Pioneer first select Fully Featured for interface operation mode on Settings and click Save.

Open Toolbox), select one of the pre-defined code snippets and press the play button to execute it.

You can edit the code or paste your own script. If after editing the code nothing happens the browser console (Ctrl + Shift + I) can help to debug the problem.

For above example to get a membership enter:

const handle = 'joystreamstats'
const hex = handle.split("").map(c => c.charCodeAt(0).toString(16).padStart(2, "0")).join("")
const memberId = await api.query.members.memberIdByHandle(`0x` + hex)
const member = await api.query.members.membershipById(memberId)
console.log(handle, memberId)

// nicely print the member object
Object.keys(member).map(key=> console.log(`${key}: ${JSON.stringify(member[key])}`))

Result:

joystreamstats 1305
registry: {}
createdAtHash: undefined
handle: "joystreamstats"
avatar_uri: "https://joystreamstats.live/favicon.ico"
about: "https://joystreamstats.live"
registered_at_block: 1
registered_at_time: 1613791830000
entry: {"genesis":null}
suspended: false
subscription: null
root_account: "5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3"
controller_account: "5C8BEb4AwVmDnxkZmbh4PVxJR5TMQ7QxU6nZtYkqTAvXhUP3"
typeDefs: {}

Useful functions:

• .hash(): Displays the storage hash
• .size(): Display the storage entry size
• .at(): Display the historic value at a given block hash (see below)
• .key: Display the key for a double map
• .keyPrefix: Display the key prefix for a double map

Subscribe to balance changes for multiple accounts:

api.query.system.account.multi([ALICE, BOB], (info) => {
  console.log('Change detected, new balances: ', info)
});

(See example Listen to multiple balances changes)

Show events between two blocks:

const start = 2000200
const end = 2000500
const hide = [ 'ExtrinsicSuccess', 'HeartbeatReceived' ]

console.log(`--- Events between ${start} and ${end} ---`)
for (let block = start ; block <= end ; ++block) {
 const hash = await api.rpc.chain.getBlockHash(block)
 api.query.system.events.at(hash).then((events) => {
  // filter hidden event methods
  const filtered = events.filter(({event}) => !hide.includes(event.method))
  if (!filtered.length) return
  console.log(`#[${block}: ${events.length} event(s)`);
  // loop through the Vec<EventRecord>
  filtered.forEach((record) => {
  // extract the phase, event and the event types
    const { event, phase } = record;
    const types = event.typeDef;
    // show what we are busy with
    console.log(` - ${event.section}.${event.method}::${phase}=${phase.toString()}`);
    console.log(event.meta.documentation.toString().padStart(`\t`));
    // loop through each of the parameters, displaying the type and data
    event.data.forEach((data, index) => {
      console.log(types[index].type + ';' + data.toString());
    })
  })
 })
}
console.log(`---`)

See augment-api-events.ts for a list of all extrinsics. Note that this runs much faster with a script connected to a local node.

• script to show era payouts

joystream-cli

Another useful even tough at first time consuming tool to inspect working groups for example is the cli:

git clone https://github.com/Joystream/joystream
cd joystream/cli
yarn
yarn link || bash # Start a new shell after linking `cli`
./bin/run api:setUri # Select the default api endpoint
./bin/run working-groups:overview
./bin/run working-groups:openings

To select another group: -g storageProviders, curators, operations[Alpha,Beta,Gamma]

api

All methods in Chain State are available using the api

To start try this compact example in joystream-api/src/general/status.ts to query the chain:

git clone https://github.com/Joystream/community-repo
cd community-repo/scripts/joystream-api
community-repo/scripts/joystream-api$ yarn
yarn install v1.22.17
[1/4] Resolving packages...
success Already up-to-date.
$ yarn build
yarn run v1.22.17
$ tsc --build tsconfig.json
Done in 5.55s.
Done in 5.76s.
/community-repo/scripts/joystream-api$ yarn status
yarn run v1.22.17
$ node lib/general/status
Chain 'Joystream' - node: Joystream Node v5.9.0-c7711424ef-x86_64-linux-gnu
Runtime Version: 9.7.0
Council size: 20
Validator count: 69
Total Validator Locked Balances: 171694540
Done in 8.75s.

If you have no local validator node, change the provider URL in line 13, for example wss://testnet-rpc-3-uk.joystream.org or wss://joystreamstats.live:9945 (alternatively click on the blue circle in Pioneer to see more, custom endpoint shows the address). Or set one up with node-setup-script :)

The folder has more examples and it is worth to look around. You can also try queries from the scripts on the JS page directly and log the output:

  const [chain, nodeName, nodeVersion] = await Promise.all([
    api.rpc.system.chain(),
    api.rpc.system.name(),
    api.rpc.system.version()
  ]);

  console.log(`Chain '${chain}' - node: ${nodeName} v${nodeVersion}`);

For the current testnet augment-api-query.ts gives a list with descriptions (also in node_modules/@joystream/types/augment) of :

/**
       * Registered unique handles and their mapping to their owner
       **/
      memberIdByHandle: AugmentedQuery<ApiType, (arg: Bytes | string | Uint8Array) => Observable<MemberId>, [Bytes]>;
      /**
       * Mapping of a controller account id to vector of member ids it controls
       **/
      memberIdsByControllerAccountId: AugmentedQuery<ApiType, (arg: AccountId | string | Uint8Array) => Observable<Vec<MemberId>>, [AccountId]>;
      /**
       * Mapping of a root account id to vector of member ids it controls.
       **/
      memberIdsByRootAccountId: AugmentedQuery<ApiType, (arg: AccountId | string | Uint8Array) => Observable<Vec<MemberId>>, [AccountId]>;
      /**
       * Mapping of member's id to their membership profile
       **/
      membershipById: AugmentedQuery<ApiType, (arg: MemberId | AnyNumber | Uint8Array) => Observable<Membership>, [MemberId]>;