tghelper package

Submodules

tghelper.tghelper module

class tghelper.tghelper.TgHelper(conn)

Bases: object

TgHelper contains helper functions for use with the pyTigerGraph library

execute_gsql(file_path)

Execute a gsql statement that is stored in a file

This method reads in the gsql statement form the filepath provided and then executes the statement using the pyTigerGraph gsql method

Parameters

file_path (str) – Path to the file that contains the gsql statement

Returns

The output received from the Tigergraph instance

upload_job(source_file, job, job_filename, lines_per_file=1000000, no_workers=5, timeout=500000)

Uses multiprocessing to upload the file and then executes the loading job for the file

This method splits the files into many files and then uses multiprocessing to individually upload and execute the loading job for each file. It uses the TgUpload class to perform these actions.

Parameters

• source_file (str) – Path to the source file containing the data for upload

• job (str) – Name of the loading job on the Tigergraph instance

• job_filename (str) – Name of the file specified in the loading job

• lines_per_file (int) – Number of lines per file

• no_workers (int) – Number of parallel workers used for multiprocessing

• timeout (int) – Timeout setting per loading job

Returns:

class tghelper.tghelper.TgUpload(source_file, tg_conn, job, job_filename, lines_per_file=1000000, no_workers=5, timeout=500000)

Bases: object

This class handles the multiprocessing, file splitting, uploading and loading job execution.

This class facilitates the execution of pyTigergraph’s runLoadingJobWithFile method using multiproccesing

producer()

The producer splits the files and creates entries into the queue for the workers

Returns:

run()

Execute the process of starting workers and producers and waiting for them to finish

Returns:

start_producers()

Start a single producer

Returns:

start_workers()

Start the number of specified workers

Returns:

worker()

The worker fetches the next file from the queue and execute the loading job

Returns:

Module contents

class tghelper.TgHelper(conn)

Bases: object

TgHelper contains helper functions for use with the pyTigerGraph library

execute_gsql(file_path)

Execute a gsql statement that is stored in a file

This method reads in the gsql statement form the filepath provided and then executes the statement using the pyTigerGraph gsql method

Parameters

file_path (str) – Path to the file that contains the gsql statement

Returns

The output received from the Tigergraph instance

upload_job(source_file, job, job_filename, lines_per_file=1000000, no_workers=5, timeout=500000)

Uses multiprocessing to upload the file and then executes the loading job for the file

This method splits the files into many files and then uses multiprocessing to individually upload and execute the loading job for each file. It uses the TgUpload class to perform these actions.

Parameters

• source_file (str) – Path to the source file containing the data for upload

• job (str) – Name of the loading job on the Tigergraph instance

• job_filename (str) – Name of the file specified in the loading job

• lines_per_file (int) – Number of lines per file

• no_workers (int) – Number of parallel workers used for multiprocessing

• timeout (int) – Timeout setting per loading job

Returns:

class tghelper.TgUpload(source_file, tg_conn, job, job_filename, lines_per_file=1000000, no_workers=5, timeout=500000)

Bases: object

This class handles the multiprocessing, file splitting, uploading and loading job execution.

This class facilitates the execution of pyTigergraph’s runLoadingJobWithFile method using multiproccesing

producer()

The producer splits the files and creates entries into the queue for the workers

Returns:

run()

Execute the process of starting workers and producers and waiting for them to finish

Returns:

start_producers()

Start a single producer

Returns:

start_workers()

Start the number of specified workers

Returns:

worker()

The worker fetches the next file from the queue and execute the loading job

Returns: