url_samples: urldbsample_extended


Introduction

Shows how to use the URL Classification using a local URL database and the update and schedule functionality in a dedicated thread.

Syntax

urldbsample_extended <dca-redist-folder> <ticket> <product> <url-list-file>
url-list-file
A file that contains several URLs (one per line). Each URL is used for the URL classification.
Note:
To get classification results you will need first to install a local URL database. This can be done by using the url_samples: urldbdownloadsample.
Incremental update files of the local URL database are downloaded by the dca::UpdateModule::performUpdate() call.
See also:

Classes

class  MyScheduleEventSubscriber
 Example implementation of a schedule event subscriber. More...

Files

file  url_samples/urldbsample_extended/linux/perform_update_thread.cpp
 

Implementation of the update thread.


file  url_samples/urldbsample_extended/linux/schedule_thread.cpp
 

Implementation of the schedule thread.


file  url_samples/urldbsample_extended/main.cpp
 

Extended URL Classification using a local URL database sample program.


file  url_samples/urldbsample_extended/mythreads.h
 

Header file for functions related to start and stop the update and schedule threads.


file  url_samples/urldbsample_extended/perform_update_thread.h
 

Header file for the update thread.


file  url_samples/urldbsample_extended/schedule_event_subscriber.h
 

Header file for the schedule event subscriber.


file  url_samples/urldbsample_extended/schedule_thread.h
 

Header file for the schedule thread.


Defines

#define DCA_BINDIR   "bin/linux"
 DCA subdirectory of the DCA binaries.
#define DCA_INITDIR   "init"
 DCA subdirectory of the DCA initialization data.
#define DCA_LOGDIR   "./logs"
 Relative directory for logfile(s).

Functions

static void SetupInitData (const std::string &redist_folder, InitData &initData)
 Sets up the given initData by substituting the given redist_folder with DCA subdirectories.
static bool StartupLibraries ()
 Initializes 3rd party library libcurl and set up open ssl callbacks to startdard implementation.
static void ShutdownLibraries ()
 Shuts down 3rd party libraries. On Windows also WSACleanup is called to shutdown Windows sockets for this process.
static void SetupLicense (const std::string &ticket, const std::string &product, LicenseData &licenseData)
 Sets up the given licenseData by copying the given ticket and product strings.
static void SetupConnectionData (DbConnectionData &cData)
 Sets up the given cData to use a local URL database.
static void PrintResults (const CategoriesInfo &catinfos, const UrlClassificationResults &cats)
 Prints out the classification results and uses the categories info for textual representation of the matched categories.
static void PrintToolHeader ()
 Prints out the name and the version of this sample.
static void PrintUsage (const char *name)
 Prints out the syntax of the sample.
static void LoadUrlFile (const std::string &fileName, std::vector< std::string > &urlList)
 Loads given fileName and adds each line to given urlList (by deleting trailing CRLFs).
void TestUrlClassification (const std::string &aUrlListFile, const DcaInstance &myDca, const UrlDbClassifier &myUrlDbClassifier, const CategoriesInfo &myCategoriesInfo)
 Performs the URL db classification withURLs found in a given text file.
int main (int argc, char *argv[])
 The main routine.
void StartupThreads (const dca::DcaInstance &aDcaInstance, const dca::UpdateModule &aUpdateModule, dca::ScheduleEventSubscriberIntf *pSubscriber)
 Starts up the update and schedule threads and supplies the given DcaInstance and UpdateModule.
void ShutdownThreads ()
 Shuts down the previously started update and schedule threads.
void mySleep (unsigned int ms)
 sleeps the given interval (in milliseconds)
void createPerformUpdateThread (const dca::UpdateModule &aUpdateModule)
 Starts up the update thread and supplies the given UpdateModule.
void shutdownPerformUpdateThread ()
 Shuts down the previously started update thread.
void createScheduleThread (const dca::DcaInstance &aDcaInstance, dca::ScheduleEventSubscriberIntf *pSubscriber)
 Starts up the schedule thread and supplies the given DcaInstance.
void shutdownScheduleThread ()
 Shuts down the previously started schedule thread.

Variables

const std::string S_UsageString
 Usage string, displayed if a parameter was missing.

Function Documentation

static void SetupInitData ( const std::string &  redist_folder,
InitData initData 
) [static]

Sets up the given initData by substituting the given redist_folder with DCA subdirectories.

Parameters:
[in] redist_folder This is the folder where the DCA has been installed to (assuming trailing fileslash)
[out] initData The InitData structure to set up
Note:
Only DCA_BINDIR differs between Windows and Linux
The directory ./logs is used for the logfile(s)

Definition at line 104 of file url_samples/urldbsample_extended/main.cpp.

static bool StartupLibraries (  )  [static]

Initializes 3rd party library libcurl and set up open ssl callbacks to startdard implementation.

On Windows its necessary to initalize Windows sockets to support IP(v6) addresses as input data.

Returns:
true if nothing fails, false only on Windows if WSAStartup returned an error

Definition at line 122 of file url_samples/urldbsample_extended/main.cpp.

static void SetupLicense ( const std::string &  ticket,
const std::string &  product,
LicenseData licenseData 
) [static]

Sets up the given licenseData by copying the given ticket and product strings.

Parameters:
[in] ticket This is the ticket data as provided with your DCA license
[in] product This is the product shortcut e.g. DC oder MS etc
[out] licenseData The LicenseData structure to set up

Definition at line 168 of file url_samples/urldbsample_extended/main.cpp.

static void SetupConnectionData ( DbConnectionData cData  )  [static]

Sets up the given cData to use a local URL database.

Parameters:
[in] cData The DbConnectionData structure to set up
Note:
Assuming the DCA has already downloaded an initial URL database. If not, set useLocalDatabase to false.

Definition at line 182 of file url_samples/urldbsample_extended/main.cpp.

static void PrintResults ( const CategoriesInfo catinfos,
const UrlClassificationResults cats 
) [static]

Prints out the classification results and uses the categories info for textual representation of the matched categories.

Parameters:
[in] catinfos The CategoriesInfo class associated with the given URL database
[in] cats The results of a URL classification

Definition at line 195 of file url_samples/urldbsample_extended/main.cpp.

static void PrintUsage ( const char *  name  )  [static]

Prints out the syntax of the sample.

Parameters:
[in] name The name of the executable

Definition at line 235 of file url_samples/urldbsample_extended/main.cpp.

static void LoadUrlFile ( const std::string &  fileName,
std::vector< std::string > &  urlList 
) [static]

Loads given fileName and adds each line to given urlList (by deleting trailing CRLFs).

Parameters:
[in] fileName - The file that contains the input URLs
[in] urlList - The list to be filled with the URLs found in fileName

Definition at line 246 of file url_samples/urldbsample_extended/main.cpp.

void TestUrlClassification ( const std::string &  aUrlListFile,
const DcaInstance myDca,
const UrlDbClassifier myUrlDbClassifier,
const CategoriesInfo myCategoriesInfo 
)

Performs the URL db classification withURLs found in a given text file.

The given aUrlListFile contains one URL per line. The URLs will be added to a vector and for each URL a URL db classification is invoked. The results are printed out by using the PrintResults() function.

Parameters:
[in] aUrlListFile The file that contains the input URLs
[in] myDca A valid set up DCA Instance
[in] myUrlDbClassifier A valid set up UrlDbClassifier
[in] myCategoriesInfo A valid set up CategoriesInfo
Note:
The results of a classification returns either "URL is unknown", "not categorized" or a "set of matched categories".
When creating a multi-threaded application you can include something similar to this function into each thread's worker function.

Definition at line 281 of file url_samples/urldbsample_extended/main.cpp.

int main ( int  argc,
char *  argv[] 
)

The main routine.

Parameters:
[in] argc The count of arguments provided
[in] argv An array of provided arguments
Returns:
5 on usage error, 10 on exception and internal error and 0 on success

Definition at line 342 of file url_samples/urldbsample_extended/main.cpp.

void StartupThreads ( const dca::DcaInstance aDcaInstance,
const dca::UpdateModule aUpdateModule,
dca::ScheduleEventSubscriberIntf pSubscriber 
)

Starts up the update and schedule threads and supplies the given DcaInstance and UpdateModule.

Parameters:
[in] aDcaInstance An initialized DcaInstance
[in] aUpdateModule An initialized UpdateModule
[in] pSubscriber A pointer to an implementation of dca::ScheduleEventSubscriberIntf

Starts up two threads

  • an update thread that calls the aUpdateModule::performUpdate() method in a loop and
  • a schedule thread that calls the aDcaInstance::schedule() method in a loop.

The called functions will check engine and content updates automatically and install them in the provided dca/init directory.

Schedule event information will be captured and handled by pSubscriber.

They will be stopped by a call to ShutdownThreads()

See also:
createPerformUpdateThread, createScheduleThread, ShutdownThreads

Definition at line 13 of file url_samples/urldbsample_extended/mythreads.cpp.

void ShutdownThreads (  ) 

Shuts down the previously started update and schedule threads.

Shutting down will stop the threads safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate any thread.

Note:
To force any running update processes to stop immediately a call to DcaInstance::signal() should be invoked.
See also:
shutdownScheduleThread, shutdownPerformUpdateThread, StartupThreads

Shutting down will stop the threads safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate any thread.

Note:
To force stop any running update processes immediately a call to DcaInstance::signal() should be invoked.
See also:
shutdownScheduleThread, shutdownPerformUpdateThread, StartupThreads

Definition at line 20 of file customdb_samples/customdbsample_extended/mythreads.cpp.

void createPerformUpdateThread ( const dca::UpdateModule aUpdateModule  ) 

Starts up the update thread and supplies the given UpdateModule.

Parameters:
aUpdateModule An initialized UpdateModule

Starts up an update thread that calls the aUpdateModule::performUpdate() method in a loop.

The called function will check engine and content updates automatically and install them in the provided dca/init directory.

It can be stopped by a call to shutdownPerformUpdateThread

Note:
There are separate implementations for Windows and Linux distributions.
See also:
shutdownPerformUpdateThread

Definition at line 67 of file url_samples/urldbsample_extended/linux/perform_update_thread.cpp.

void shutdownPerformUpdateThread (  ) 

Shuts down the previously started update thread.

Shutting down will stop the thread safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate the thread.

Note:
To force any running update processes to stop immediately a call to DcaInstance::signal() should be invoked.
There are separate implementations for Windows and Linux distributions.
See also:
createPerformUpdateThread

Definition at line 71 of file customdb_samples/customdbsample_extended/linux/perform_update_thread.cpp.

void createScheduleThread ( const dca::DcaInstance aDcaInstance,
dca::ScheduleEventSubscriberIntf pSubscriber 
)

Starts up the schedule thread and supplies the given DcaInstance.

Parameters:
[in] aDcaInstance An initialized DcaInstance
[in] pSubscriber A pointer to an implementation of dca::ScheduleEventSubscriberIntf

Start up a schedule thread that calls the aDcaInstance::schedule() method in a loop.

It can be stopped by a call to ShutdownScheduleThread()

Schedule event information will be captured and handled by pSubscriber.

Note:
There are separate implementations for Windows and Linux distributions.
See also:
shutdownScheduleThread

Definition at line 67 of file url_samples/urldbsample_extended/linux/schedule_thread.cpp.

void shutdownScheduleThread (  ) 

Shuts down the previously started schedule thread.

Shutting down will stop the thread safely.

If an update is already in progress the function will return as soon as possible but will not force-terminate any thread.

Note:
To force any running update processes to stop immediately a call to DcaInstance::signal() should be invoked.
There are separate implementations for Windows and Linux distributions.
See also:
createScheduleThread

Definition at line 66 of file customdb_samples/customdbsample_extended/linux/schedule_thread.cpp.


Variable Documentation

const std::string S_UsageString
Initial value:
        " <redist-folder> <ticket> <product> <url-list-file>\n"
        "  redist-folder - the folder where the DCA is installed to\n"
        "  ticket - a valid ticket\n"
        "  product - the product associated with your ticket\n"
        "  url-list-file - file that includes the URLs to classify\n\n"

Usage string, displayed if a parameter was missing.

Definition at line 67 of file url_samples/urldbsample_extended/main.cpp.


Generated on 26 Sep 2016 for dca_interface by  doxygen 1.6.1