grilly.datasets.merge_svc

SVC Dataset Merger

Merges instruct_svc_semantic.jsonl and conversations_svc_cleaned.jsonl into a single shuffled training file.

Functions

load_jsonl(filepath)	Load all lines from a JSONL file as raw strings.
main()	Autogenerated reference for grilly.datasets.merge_svc.main.

Classes

Counter([iterable])	Dict subclass for counting hashable items.
Path(*args, **kwargs)	PurePath subclass that can make system calls.

grilly.datasets.merge_svc.load_jsonl(filepath)

Load all lines from a JSONL file as raw strings.

Dependencies: None detected from callable globals.

Variables: filepath (pathlib.Path, required).

Usage Example

from grilly.datasets.merge_svc import load_jsonl

result = load_jsonl(filepath=None)

Parameters

filepath (Path) –

Return type

list[str]

grilly.datasets.merge_svc.main()

Autogenerated reference for grilly.datasets.merge_svc.main.

Dependencies: collections, json, pathlib, random, sys.

Variables: This callable does not take explicit input variables.

Usage Example

from grilly.datasets.merge_svc import main

result = main()