Class: Extension Abstract

Defined in: src/kermit/Extension.coffee
Inherits: Mixin

Overview

This class is abstract.
Note: The mapping implicitly associates each extension with the ExtensionPoint corresponding to one of ProcessingPhase.ALL
Note: Each extension may expose only one handler per phase value

Extensions are the core abstraction for adding actual item processing functionality to the Crawler. In fact, all of the available item processing functionality like filtering, queueing, streaming, logging etc. is implemented by means of extensions.

A major motivation of the extension design is to support the principles of separation of concern as well as single responsibility. It aims to encourage the development of relatively small, testable and reusable item processing components.

Each extension can expose handlers for item processing by mapping them to one of the defined values of ProcessingPhase.

See Crawler for the state diagram modeling the values and transitions of ProcessingPhase and respective ExtensionPoints.

See also:

Direct Known Subclasses

ExtensionPointConnector Spooler Completer RequestItemMapper Cleanup QueueConnector QueueWorker RequestStreamer RandomizedDelay RemoteControl AutoShutdown ResourceDiscovery Histogrammer Monitoring OfflineStorage OfflineServer ResultVerification TransitionRecorder RejectingExtension ResponseStreamLogger

Instance Method Summary

Constructor Details

# (void) constructor(handlers = {})

Construct a new extension. By convention the property "name" will be assigned the class name of this extension to handlers that will be invoked for items with that phase

Parameters:

Instance Method Details

# (void) initialize(context)

This method is called by the corresponding ExtensionPoint during crawler construction.

Parameters:

Throws:

  • ( ? ) — Error if it does not find the context to be providing what it expects.

Pretty print an object

# (void) merge(a, b)

Merge two objects recursively. This is used to combine user specified options with default options

# (void) targets()

Get all ProcessingPhase values handled by this extension

# (void) verify()

Run validity checks of this extension. Called after initialization and before actual item processing starts

Throws:

  • ( ? ) — Error if the configuration is invalid in any way

# (void) onStart(fn)

Register a callback function to be invoked when start command is executed

# (void) onStop(fn)

# (String) toString()

Returns:

  • ( String ) — Human readable description of this extension

    Quickly fuzzy find classes, mixins, methods, file:

    Control the navigation frame:

    You can focus and blur the search input: