Getting started with JuiceFS using TiKV database

TiKV

Install TiKV

1. Install TiUP

$ curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
$ source .bash_profile
$ tiup -v 
1.5.4 tiup
Go Version: go1.16.6
Git Ref: v1.5.4
GitHash: b629670276269cd1518eb28f362a5180135cc985

2. Deploy TiKV cluster

$ tiup playground --mode tikv-slim
PD client endpoints: [127.0.0.1:2379]
To view the Prometheus: http://127.0.0.1:9090
To view the Grafana: http://127.0.0.1:3000

Install JuiceFS

$ JFS_LATEST_TAG=$(curl -s https://api.github.com/repos/juicedata/juicefs/releases/latest | grep 'tag_name' | cut -d '"' -f 4 | tr -d 'v')
$ wget "https://github.com/juicedata/juicefs/releases/download/v${JFS_LATEST_TAG}/juicefs-${JFS_LATEST_TAG}-linux-amd64.tar.gz"
$ mkdir juice && tar -zxvf "juicefs-${JFS_LATEST_TAG}-linux-amd64.tar.gz" -C juice
$ sudo install juice/juicefs /usr/local/bin
$ juicefs

NAME:
juicefs - A POSIX file system built on Redis and object storage.

USAGE:
juicefs [global options] command [command options] [arguments...]

VERSION:
0.16.1 (2021-08-16 2edcfc0)

COMMANDS:
format format a volume
mount mount a volume
umount unmount a volume
gateway S3-compatible gateway
sync sync between two storage
rmr remove directories recursively
info show internal information for paths or inodes
bench run benchmark to read/write/stat big/small files
gc collect any leaked objects
fsck Check consistency of file system
profile analyze access log
stats show runtime stats
status show status of JuiceFS
warmup build cache for target directories/files
dump dump metadata into a JSON file
load load metadata from a previously dumped JSON file
help, h Shows a list of commands or help for one command

GLOBAL OPTIONS:
--verbose, --debug, -v enable debug log (default: false)
--quiet, -q only warning and errors (default: false)
--trace enable trace log (default: false)
--no-agent Disable pprof (:6060) and gops (:6070) agent (default: false)
--help, -h show help (default: false)
--version, -V print only the version (default: false)

COPYRIGHT:
AGPLv3

Usage

1. Create a file system

$ juicefs format \
--storage minio \
--bucket http://127.0.0.1:9000/mystor \
--access-key minioadmin \
--secret-key minioadmin \
tikv://127.0.0.1:2379/mystor \
mystor
  • --storage: Specify the data storage engine, here is minio.
  • --bucket: Specify the bucket access URL. The bucket named mystor I created in advance on MinIO.
  • --access-key and --secret-key: Specify the secret key for accessing the object storage service API.
  • When using TiKV to store metadata, set the PD address of the cluster. When multiple file systems or applications share the same TiKV, it is recommended to add an optional prefix, where the prefix mystor is specified in the PD address.
2021/08/12 23:28:36.932241 juicefs[101222] <INFO>: Meta address: tikv://127.0.0.1:2379/mystor
[2021/08/12 23:28:36.932 +08:00] [INFO] [client.go:214] ["[pd] create pd client with endpoints"] [pd-address="[127.0.0.1:2379]"]
[2021/08/12 23:28:36.935 +08:00] [INFO] [base_client.go:346] ["[pd] switch leader"] [new-leader=http://127.0.0.1:2379] [old-leader=]
[2021/08/12 23:28:36.935 +08:00] [INFO] [base_client.go:126] ["[pd] init cluster id"] [cluster-id=6995548759432331426]
[2021/08/12 23:28:36.935 +08:00] [INFO] [client.go:238] ["[pd] create tso dispatcher"] [dc-location=global]
2021/08/12 23:28:36.936892 juicefs[101222] <INFO>: Data uses minio://127.0.0.1:9000/mystor/mystor/
2021/08/12 23:28:36.976722 juicefs[101222] <INFO>: Volume is formatted as {Name:mystor UUID:0c9594a8-fe2c-463c-a4b6-eb815f38c843 Storage:minio Bucket:http://127.0.0.1:9000/mystor AccessKey:minioadmin SecretKey:removed BlockSize:4096 Compression:none Shards:0 Partitions:0 Capacity:0 Inodes:0 EncryptKey:}

2. Mount the file system

$ sudo juicefs mount -d tikv://127.0.0.1:2379/mystor ~/jfs
2021/08/12 23:34:44.288136 juicefs[101873] <INFO>: Meta address: tikv://127.0.0.1:2379/mystor
[2021/08/12 23:34:44.288 +08:00] [INFO] [client.go:214] ["[pd] create pd client with endpoints"] [pd-address="[127.0.0.1:2379]"]
[2021/08/12 23:34:44.291 +08:00] [INFO] [base_client.go:346] ["[pd] switch leader"] [new-leader=http://127.0.0.1:2379] [old-leader=]
[2021/08/12 23:34:44.291 +08:00] [INFO] [base_client.go:126] ["[pd] init cluster id"] [cluster-id=6995548759432331426]
[2021/08/12 23:34:44.291 +08:00] [INFO] [client.go:238] ["[pd] create tso dispatcher"] [dc-location=global]
2021/08/12 23:34:44.296270 juicefs[101873] <INFO>: Data use minio://127.0.0.1:9000/mystor/mystor/
2021/08/12 23:34:44.296768 juicefs[101873] <INFO>: Disk cache (/var/jfsCache/0c9594a8-fe2c-463c-a4b6-eb815f38c843/): capacity (1024 MB), free ratio (10%), max pending pages (15)
2021/08/12 23:34:44.800551 juicefs[101873] <INFO>: OK, mystor is ready at /home/herald/jfs
$ df -Th
File system type capacity used usable used% mount point
JuiceFS:mystor fuse.juicefs 1.0P 64K 1.0P 1% /home/herald/jfs

3. View file system information

$ juicefs status tikv://127.0.0.1:2379/mystor
{
"Setting": {
"Name": "mystor",
"UUID": "9f50f373-a7ec-4d5b-b790-3defbf6d0509",
"Storage": "minio",
"Bucket": "http://127.0.0.1:9000/mystor",
"AccessKey": "minioadmin",
"SecretKey": "removed",
"BlockSize": 4096,
"Compression": "none",
"Shards": 0,
"Partitions": 0,
"Capacity": 0,
"Inodes": 0
},
"Sessions": [
{
"Sid": 2,
"Heartbeat": "2021-08-13T10:43:35+08:00",
"Version": "0.16-dev (2021-08-12 a871c3d)",
"Hostname": "herald-manjaro",
"MountPoint": "/home/herald/jfs",
"ProcessID": 6309
}
]
}
$ sudo cat ~/jfs/.config
{
"Meta": {
"Strict": true,
"Retries": 10,
"CaseInsensi": false,
"ReadOnly": false,
"OpenCache": 0,
"MountPoint": "jfs",
"Subdir": ""
},
"Format": {
"Name": "myabc",
"UUID": "e9d8373c-7ced-49d9-a033-75f6abb44854",
"Storage": "minio",
"Bucket": "http://127.0.0.1:9000/mystor",
"AccessKey": "minioadmin",
"SecretKey": "removed",
"BlockSize": 4096,
"Compression": "none",
"Shards": 0,
"Partitions": 0,
"Capacity": 0,
"Inodes": 0
},
"Chunk": {
"CacheDir": "/var/jfsCache/e9d8373c-7ced-49d9-a033-75f6abb44854",
"CacheMode": 384,
"CacheSize": 1024,
"FreeSpace": 0.1,
"AutoCreate": true,
"Compress": "none",
"MaxUpload": 20,
"Writeback": false,
"Partitions": 0,
"BlockSize": 4194304,
"GetTimeout": 60000000000,
"PutTimeout": 60000000000,
"CacheFullBlock": true,
"BufferSize": 314572800,
"Readahead": 0,
"Prefetch": 1
},
"Version": "0.16.1 (2021-08-16 2edcfc0)",
"Mountpoint": "jfs"
}

4. Unmount the file system

$ sudo juicefs umount ~/jfs

5. Mount at boot

$ sudo cp /usr/local/bin/juicefs /sbin/mount.juicefs
tikv://127.0.0.1:2379/mystor    /home/herald/jfs       juicefs     _netdev,cache-size=20480     0  0

Summarize

Open Source Contribution Guide

Things you can do for the community:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store