org.apache.nutch.crawl

Class GeneratorReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<K1,V1,K2,V2>

org.apache.gora.mapreduce.GoraReducer<GeneratorJob.SelectorEntry,WebPage,java.lang.String,WebPage>

org.apache.nutch.crawl.GeneratorReducer

public class GeneratorReducer
extends org.apache.gora.mapreduce.GoraReducer<GeneratorJob.SelectorEntry,WebPage,java.lang.String,WebPage>

Reduce class for generate The #reduce() method write a random integer to all generated URLs. This random number is then used by FetcherJob.FetcherMapper.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

Reducer.Context

Constructor Summary

Constructors

Constructor and Description
GeneratorReducer()

Method Summary

Modifier and Type	Method and Description
protected void	reduce(GeneratorJob.SelectorEntry key, java.lang.Iterable<WebPage> values, Reducer.Context context)
protected void	setup(Reducer.Context context)

Methods inherited from class org.apache.gora.mapreduce.GoraReducer

initReducerJob, initReducerJob, initReducerJob

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GeneratorReducer

public GeneratorReducer()

Method Detail

reduce

protected void reduce(GeneratorJob.SelectorEntry key,
                      java.lang.Iterable<WebPage> values,
                      Reducer.Context context)
               throws java.io.IOException,
                      java.lang.InterruptedException

Overrides:

reduce in class Reducer<GeneratorJob.SelectorEntry,WebPage,java.lang.String,WebPage>

Throws:

java.io.IOException

java.lang.InterruptedException

setup

protected void setup(Reducer.Context context)
              throws java.io.IOException,
                     java.lang.InterruptedException

Overrides:

setup in class Reducer<GeneratorJob.SelectorEntry,WebPage,java.lang.String,WebPage>

Throws:

java.io.IOException

java.lang.InterruptedException