url_samples: urldbsample_extended

Introduction

Shows how to use the URL Classification using a local URL database and the update and schedule functionality in a dedicated thread.

Syntax

urldbsample_extended <dca-redist-folder> <ticket> <product> <url-list-file>

: url-list-file

: A file that contains several URLs (one per line). Each URL is used for the URL classification.

Note:: To get classification results you will need first to install a local URL database. This can be done by using the url_samples: urldbdownloadsample.
Incremental update files of the local URL database are downloaded by the dca::UpdateModule::performUpdate() call.

See also:

dca::DcaInstance, dca::License, dca::DbConnection
dca::UrlClassification, dca::UrlDbClassifier
dca::UrlClassificationResults
dca::UpdateModule, dca::UpdateModule::performUpdate()
dca::ScheduleEventSubscriberIntf, dca::DcaInstance::schedule()
url_samples: urldbsample
url_samples: urldbsample_remote
Samples Main Documentation: Samples

Classes
class	MyScheduleEventSubscriber
	Example implementation of a schedule event subscriber. More...
Files
file	url_samples/urldbsample_extended/linux/perform_update_thread.cpp
	Implementation of the update thread.
file	url_samples/urldbsample_extended/linux/schedule_thread.cpp
	Implementation of the schedule thread.
file	url_samples/urldbsample_extended/main.cpp
	Extended URL Classification using a local URL database sample program.
file	url_samples/urldbsample_extended/mythreads.h
	Header file for functions related to start and stop the update and schedule threads.
file	url_samples/urldbsample_extended/perform_update_thread.h
	Header file for the update thread.
file	url_samples/urldbsample_extended/schedule_event_subscriber.h
	Header file for the schedule event subscriber.
file	url_samples/urldbsample_extended/schedule_thread.h
	Header file for the schedule thread.
Defines
#define	DCA_BINDIR "bin/linux"
	DCA subdirectory of the DCA binaries.
#define	DCA_INITDIR "init"
	DCA subdirectory of the DCA initialization data.
#define	DCA_LOGDIR "./logs"
	Relative directory for logfile(s).
Functions
static void	SetupInitData (const std::string &redist_folder, InitData &initData)
	Sets up the given initData by substituting the given redist_folder with DCA subdirectories.
static bool	StartupLibraries ()
	Initializes 3rd party library libcurl and set up open ssl callbacks to startdard implementation.
static void	ShutdownLibraries ()
	Shuts down 3rd party libraries. On Windows also WSACleanup is called to shutdown Windows sockets for this process.
static void	SetupLicense (const std::string &ticket, const std::string &product, LicenseData &licenseData)
	Sets up the given licenseData by copying the given ticket and product strings.
static void	SetupConnectionData (DbConnectionData &cData)
	Sets up the given cData to use a local URL database.
static void	PrintResults (const CategoriesInfo &catinfos, const UrlClassificationResults &cats)
	Prints out the classification results and uses the categories info for textual representation of the matched categories.
static void	PrintToolHeader ()
	Prints out the name and the version of this sample.
static void	PrintUsage (const char *name)
	Prints out the syntax of the sample.
static void	LoadUrlFile (const std::string &fileName, std::vector< std::string > &urlList)
	Loads given fileName and adds each line to given urlList (by deleting trailing CRLFs).
void	TestUrlClassification (const std::string &aUrlListFile, const DcaInstance &myDca, const UrlDbClassifier &myUrlDbClassifier, const CategoriesInfo &myCategoriesInfo)
	Performs the URL db classification withURLs found in a given text file.
int	main (int argc, char *argv[])
	The main routine.
void	StartupThreads (const dca::DcaInstance &aDcaInstance, const dca::UpdateModule &aUpdateModule, dca::ScheduleEventSubscriberIntf *pSubscriber)
	Starts up the update and schedule threads and supplies the given DcaInstance and UpdateModule.
void	ShutdownThreads ()
	Shuts down the previously started update and schedule threads.
void	mySleep (unsigned int ms)
	sleeps the given interval (in milliseconds)
void	createPerformUpdateThread (const dca::UpdateModule &aUpdateModule)
	Starts up the update thread and supplies the given UpdateModule.
void	shutdownPerformUpdateThread ()
	Shuts down the previously started update thread.
void	createScheduleThread (const dca::DcaInstance &aDcaInstance, dca::ScheduleEventSubscriberIntf *pSubscriber)
	Starts up the schedule thread and supplies the given DcaInstance.
void	shutdownScheduleThread ()
	Shuts down the previously started schedule thread.
Variables
const std::string	S_UsageString
	Usage string, displayed if a parameter was missing.

Function Documentation

static void SetupInitData	(	const std::string &	redist_folder,
		InitData &	initData
	)			`[static]`

Sets up the given initData by substituting the given redist_folder with DCA subdirectories.

Parameters:

`[in]`	redist_folder	This is the folder where the DCA has been installed to (assuming trailing fileslash)
`[out]`	initData	The InitData structure to set up

Note:: Only DCA_BINDIR differs between Windows and Linux; The directory ./logs is used for the logfile(s)

Definition at line 104 of file url_samples/urldbsample_extended/main.cpp.

static bool StartupLibraries ( ) [static]

Initializes 3rd party library libcurl and set up open ssl callbacks to startdard implementation.

On Windows its necessary to initalize Windows sockets to support IP(v6) addresses as input data.

Returns:: true if nothing fails, false only on Windows if WSAStartup returned an error

Definition at line 122 of file url_samples/urldbsample_extended/main.cpp.

static void SetupLicense	(	const std::string &	ticket,
		const std::string &	product,
		LicenseData &	licenseData
	)			`[static]`

Sets up the given licenseData by copying the given ticket and product strings.

Parameters:

`[in]`	ticket	This is the ticket data as provided with your DCA license
`[in]`	product	This is the product shortcut e.g. DC oder MS etc
`[out]`	licenseData	The LicenseData structure to set up

Definition at line 168 of file url_samples/urldbsample_extended/main.cpp.

static void SetupConnectionData ( DbConnectionData & cData ) [static]

Sets up the given cData to use a local URL database.

Parameters:

[in] cData The DbConnectionData structure to set up

Note:: Assuming the DCA has already downloaded an initial URL database. If not, set useLocalDatabase to false.

Definition at line 182 of file url_samples/urldbsample_extended/main.cpp.

static void PrintResults	(	const CategoriesInfo &	catinfos,
		const UrlClassificationResults &	cats
	)			`[static]`

Prints out the classification results and uses the categories info for textual representation of the matched categories.

Parameters:

`[in]`	catinfos	The CategoriesInfo class associated with the given URL database
`[in]`	cats	The results of a URL classification

Definition at line 195 of file url_samples/urldbsample_extended/main.cpp.

static void PrintUsage ( const char * name ) [static]

Prints out the syntax of the sample.

Parameters:

[in] name The name of the executable

Definition at line 235 of file url_samples/urldbsample_extended/main.cpp.

static void LoadUrlFile	(	const std::string &	fileName,
		std::vector< std::string > &	urlList
	)			`[static]`

Loads given fileName and adds each line to given urlList (by deleting trailing CRLFs).

Parameters:

`[in]`	fileName	- The file that contains the input URLs
`[in]`	urlList	- The list to be filled with the URLs found in fileName

Definition at line 246 of file url_samples/urldbsample_extended/main.cpp.

void TestUrlClassification	(	const std::string &	aUrlListFile,
		const DcaInstance &	myDca,
		const UrlDbClassifier &	myUrlDbClassifier,
		const CategoriesInfo &	myCategoriesInfo
	)

Performs the URL db classification withURLs found in a given text file.

The given aUrlListFile contains one URL per line. The URLs will be added to a vector and for each URL a URL db classification is invoked. The results are printed out by using the PrintResults() function.

Parameters:

`[in]`	aUrlListFile	The file that contains the input URLs
`[in]`	myDca	A valid set up DCA Instance
`[in]`	myUrlDbClassifier	A valid set up UrlDbClassifier
`[in]`	myCategoriesInfo	A valid set up CategoriesInfo

Note:: The results of a classification returns either "URL is unknown", "not categorized" or a "set of matched categories".; When creating a multi-threaded application you can include something similar to this function into each thread's worker function.

Definition at line 281 of file url_samples/urldbsample_extended/main.cpp.

int main	(	int	argc,
		char *	argv[]
	)

The main routine.

Parameters:

`[in]`	argc	The count of arguments provided
`[in]`	argv	An array of provided arguments

Returns:: 5 on usage error, 10 on exception and internal error and 0 on success

Definition at line 342 of file url_samples/urldbsample_extended/main.cpp.

void StartupThreads	(	const dca::DcaInstance &	aDcaInstance,
		const dca::UpdateModule &	aUpdateModule,
		dca::ScheduleEventSubscriberIntf *	pSubscriber
	)

Starts up the update and schedule threads and supplies the given DcaInstance and UpdateModule.

Parameters:

`[in]`	aDcaInstance	An initialized DcaInstance
`[in]`	aUpdateModule	An initialized UpdateModule
`[in]`	pSubscriber	A pointer to an implementation of dca::ScheduleEventSubscriberIntf

Starts up two threads

an update thread that calls the aUpdateModule::performUpdate() method in a loop and
a schedule thread that calls the aDcaInstance::schedule() method in a loop.

The called functions will check engine and content updates automatically and install them in the provided dca/init directory.

Schedule event information will be captured and handled by pSubscriber.

They will be stopped by a call to ShutdownThreads()

See also:: createPerformUpdateThread, createScheduleThread, ShutdownThreads

Definition at line 13 of file url_samples/urldbsample_extended/mythreads.cpp.

void ShutdownThreads ( )

Shuts down the previously started update and schedule threads.

Shutting down will stop the threads safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate any thread.

Note:: To force any running update processes to stop immediately a call to DcaInstance::signal() should be invoked.

See also:: shutdownScheduleThread, shutdownPerformUpdateThread, StartupThreads

Shutting down will stop the threads safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate any thread.

Note:: To force stop any running update processes immediately a call to DcaInstance::signal() should be invoked.

See also:: shutdownScheduleThread, shutdownPerformUpdateThread, StartupThreads

Definition at line 20 of file customdb_samples/customdbsample_extended/mythreads.cpp.

void createPerformUpdateThread ( const dca::UpdateModule & aUpdateModule )

Starts up the update thread and supplies the given UpdateModule.

Parameters:

aUpdateModule

An initialized UpdateModule

Starts up an update thread that calls the aUpdateModule::performUpdate() method in a loop.

The called function will check engine and content updates automatically and install them in the provided dca/init directory.

It can be stopped by a call to shutdownPerformUpdateThread

Note:: There are separate implementations for Windows and Linux distributions.

See also:: shutdownPerformUpdateThread

Definition at line 67 of file url_samples/urldbsample_extended/linux/perform_update_thread.cpp.

void shutdownPerformUpdateThread ( )

Shuts down the previously started update thread.

Shutting down will stop the thread safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate the thread.

Note:: To force any running update processes to stop immediately a call to DcaInstance::signal() should be invoked.; There are separate implementations for Windows and Linux distributions.

See also:: createPerformUpdateThread

Definition at line 71 of file customdb_samples/customdbsample_extended/linux/perform_update_thread.cpp.

void createScheduleThread	(	const dca::DcaInstance &	aDcaInstance,
		dca::ScheduleEventSubscriberIntf *	pSubscriber
	)

Starts up the schedule thread and supplies the given DcaInstance.

Parameters:

`[in]`	aDcaInstance	An initialized DcaInstance
`[in]`	pSubscriber	A pointer to an implementation of dca::ScheduleEventSubscriberIntf

Start up a schedule thread that calls the aDcaInstance::schedule() method in a loop.

It can be stopped by a call to ShutdownScheduleThread()

Schedule event information will be captured and handled by pSubscriber.

Note:: There are separate implementations for Windows and Linux distributions.

See also:: shutdownScheduleThread

Definition at line 67 of file url_samples/urldbsample_extended/linux/schedule_thread.cpp.

void shutdownScheduleThread ( )

Shuts down the previously started schedule thread.

Shutting down will stop the thread safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate any thread.

Note:: To force any running update processes to stop immediately a call to DcaInstance::signal() should be invoked.; There are separate implementations for Windows and Linux distributions.

See also:: createScheduleThread

Definition at line 66 of file customdb_samples/customdbsample_extended/linux/schedule_thread.cpp.

Variable Documentation

const std::string S_UsageString

Initial value:

        " <redist-folder> <ticket> <product> <url-list-file>\n"
        "  redist-folder - the folder where the DCA is installed to\n"
        "  ticket - a valid ticket\n"
        "  product - the product associated with your ticket\n"
        "  url-list-file - file that includes the URLs to classify\n\n"

Usage string, displayed if a parameter was missing.

Definition at line 67 of file url_samples/urldbsample_extended/main.cpp.

url_samples: urldbsample_extended

Introduction

Classes

Files

Defines

Functions

Variables

Function Documentation

Variable Documentation