Skip to content

metaid-developers/man-indexer

Repository files navigation

Introduction

The MetaID App Node is an open source backend indexer that accompanies MetaID, which makes it convenient to synchronize MetaID data and asset protocols. MAN is the abbreviation of MetaID App Node. In this document, it specifically refers to the data indexer for UTXO chain applications developed based on the MetaID protocol.

Main features of MAN

  1. Support all UTXO model blockchains
  2. Discover MetaID protocol data in block order and transaction order, while supporting data indexing in the memory pool
  3. Out of the box, it supports multiple database adapters, such as mongodb, elasticsearch, mariadb, postgresql, etc. Developers can choose according to the application blockchain.
  4. Developer-friendly, for common data applications, it provides a general-purpose data query API. In addition, for complex data, MAN plans to implement Graph Query Language.
  5. Controllable indexed data volume, MAN supports multiple data synchronization modes, such as full data synchronization, single application data synchronization, and multiple application combination synchronization. Developers can obtain the data they need through simple configuration.

Application development process based on MAN

Download the MAN program source code to compile, or directly download the latest MAN-Release program Modify the relevant configuration file Run MAN Use MAN Api and MetaID SDK for development Carry out development and debugging on the test network of the relevant UTXO chain Mainnet release

Build and Run

Dependencies

1. libzmq

The man indexer memory pool data depends on zmq, and libzmq needs to be installed before compilation, otherwise the compilation will fail.

OSX

brew install zmq

Linux

Fedora

dnf install zeromq-devel

Ubuntu/Debian/Mint

apt-get install libzmq3-dev

For more information, please refer to https://zeromq.org/download/

2. golang version

MAN's main development language is golang, and go >= 1.20 is required

Build

go mod tidy
go build 

Configuration file

The configuration file is in the same directory as the program running file, and the name must be config.toml

[sync]
syncAllData = true  //Whether to save the full amount of pin data
syncBeginTime = ""  //Synchronization data start time
syncEndTime = ""    //Synchronization data end time, empty means synchronization all the time
syncProtocols = ["payLike"] //Specify the specific protocol to be synchronized, which can be multiple
[protocols]
  [protocols.payLike] //Specific protocol data structure
  //Field name
  fields = [{name = "isLike",class = "string",length = 1},
           {name = "likeTo",class = "string",length = 100}]
  //Protocol database index
  indexes = [{fields = ["likeTo"],unique = false},
             {fields = ["pinId"],unique = true},
             {fields = ["pinNumber"],unique = false},
             {fields = ["pinAddress"],unique = false},
            ]
//UTXO chain configuration
[btc]
initialHeight = 2570900 //Default starting block height
//RPC related configuration
rpcHost = "127.0.0.1:18332"
rpcUser = "test"
rpcPass = "test"
rpcHttpPostMode = true
rpcDisableTLS = true
zmqHost = "tcp://127.0.0.1:28336"
//Database configuration
[mongodb]
mongoURI = "mongodb://root:[email protected]:27017"
dbName = "man_testnet"
poolSize = 200
timeOut = 20
//Web configuration
[web]
port = ":7777" //Port
//ssl configuration, default to http access if empty
pemFile = ""  //ssl certificate, pem file path
keyFile = ""  //ssl certificate, key file path
Run
./manindexer -chain=btc -databse=mongo -server=1 -test=1

Browser

The MAN indexer comes with a built-in MetaID browser that supports MetaID-related data queries. This is the deployed and live MAN browser: https://man.metaid.io

Running

We support the following three deployment and execution methods.

  1. Compile and run

  • Compile according to the documentation's compilation section.
  • Properly configure the config.toml file in the same directory as the executable.
  • Run the executable.
./manindexer -chain=btc -databse=mongo -server=1 -test=1
  1. Download the release

  • Download the latest release from here.
  • Extract the files.
  • Properly configure the config.toml file in the same directory as the executable.
  • Run the executable.
  1. Docker

  • You need to install Docker and Docker Compose first.
  • Download docker-compose.yml from here
  • Download .env from here
  • Fill in your configuration items in the .env file, especially the Bitcoin RPC node information.
  • Run docker-compose up -d on the server.
  • You can open the MAN browser using http://DOMAIN_NAME:SERVER_PORT as configured in the .env file, for example, http://xxx.com:7778.
./run_manindex.sh

To start an HTTP web service, specify the server parameter as 1 when running the program.

./manindexer -server=1

Parameters

-chain string
        Which chain to perform indexing (default "btc")
  -database string
        Which database to use (default "mongo")
  -server string
        Run the explorer service (default "1")
  -test string
        Connect to testnet (default "0")

The optional values for 'test' are 0, 1, and 2, corresponding to mainnet, testnet3, and regtest networks, respectively.

The service's default port is 80/443. If a specific port needs to be specified, it can be done through the port setting under the web category in the configuration file.

After starting the service, access it via http://127.0.0.0:{port}.

Browser Features

Search

Accepts keywords such as MetaID, MetaID Number, Pin Id for querying, but does not support fuzzy search.

PIN

Lists all PINs in reverse chronological order. Clicking on a PIN allows you to view more information, such as: 9bc429654d35a11e5dde0136e3466faa03507d7377769743fafa069e38580243i0

MetaID

Lists all MetaIDs in descending order of creation time. Clicking on a MetaID allows you to view the PIN that created it.

Block

Lists all blocks with MetaID protocol data in reverse order of block height. Clicking on a card allows you to view the specific transaction details in that block, such as: https://man.metaid.io/block/844453

Mempool

Lists MetaID data in the memory pool, which is automatically deleted after being included in a block.

Cli-Wallet

MAN has a built-in BTC wallet that can connect to a BTC node using the command line to create a wallet and generate an address. It supports balance and UTXO queries for this address, as well as MRC20 related operation. Special note: MAN Wallet requires local node RPC support and authentication information for the node, such as RpcUser and RpcPass.

Cli

Enter the MAN Indexer directory in the terminal and run

/man-cli help

cli-help

This is a MAN-CLI, which is a tool to interact with metaid-v2 in bitcoin chain
Usage:
  man-cli [flags]
  man-cli [command]

Available Commands:
  completion   Generate the autocompletion script for the specified shell
  getbalance   Display balance
  help         Help about any command
  init-wallet  Initialize MAN wallet
  mrc20balance Display the MRC20 balance of the default address
  mrc20op      MRC20 related operation commands,support deploy, mint, transfer
  utxo         Display utxo list
  version      version subcommand show git version info.

Flags:
  -h, --help   help for man-cli

JSON API

Basic API

Here is the deployed public API service: https://man.metaid.io

  1. JSON API ‐ PIN
  2. JSON API ‐ MetaID
  3. JSON API ‐ Follow
  4. JSON API ‐ MRC20

generalQuery

General query for protocols data, supports fetching data using get, count, sum methods.

Endpoint: /api/generalQuery

Method: POST

{
    "collection": "Name of the collection to query, required, e.g.,: pins",
    "action": "Query operation, supports get, count, sum , e.g.,: sum",
    "filterRelation": "Query condition relationship, supports or, and (cannot be mixed), e.g.,: or", 
    "field": [ "Field to return in the query, required for sum operation, e.g.,: number"], 
    "filter": [
	       { 
	         "operator": " Condition operator, supports =, >, >=, <, <=, e.g.,: =", 
	         "key": "Condition field,e.g.,:number",
	         "value": 1 
	       },
	       {
	         "operator": "=", "key": "number", "value": 2
	        }
          ],
	"cursor": 0,
	"limit": 1, 
	"sort": [ 
        	"number", 
		"desc"
        ]
}

Successful Response Example

{
	"collection": "paylike",
	"action": "get",
	"filterRelation": "and",
	"field": [],
	"filter": [{
		"operator": "=",
		"key": "likeTo",
		"value": "9fec9e5eb879049bd8403ffa45ca0e2756b6c14434b507ccdaf7771d5ec4edf9i0"
	}],
	"cursor": 0,
	"limit": 99999,
	"sort": []
}

Failed Response Example

{
    "code": -1,
    "message": "Data not found",
    "data": null
}