[TOC]

org.apache.nutch.crawl

Class CrawlDbReducer

    • All Implemented Interfaces:
    • Closeable, AutoCloseable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Reducer 
public class CrawlDbReducer
extends Object
implements org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.Text,CrawlDatum,org.apache.hadoop.io.Text,CrawlDatum>

Merge new page entries with existing entries.

Field Summary

Fields Modifier and Type Field and Description static org.slf4j.Logger LOG

Constructor Summary

Constructors Constructor and Description CrawlDbReducer()

Method Summary

Methods Modifier and Type Method and Description void close() void configure(org.apache.hadoop.mapred.JobConf job) void reduce(org.apache.hadoop.io.Text key, Iterator values, org.apache.hadoop.mapred.OutputCollector output, org.apache.hadoop.mapred.Reporter reporter) 

-

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

-

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

-

CrawlDbReducer

public CrawlDbReducer()

Method Detail

-

configure

public void configure(org.apache.hadoop.mapred.JobConf job)
  - Specified by: 
  - <code>configure</code> in interface <code>org.apache.hadoop.mapred.JobConfigurable</code>        
-

close

public void close()
  - Specified by: 
  - <code>close</code> in interface <code>Closeable</code> 
  - Specified by: 
  - <code>close</code> in interface <code>AutoCloseable</code>        
-

reduce

public void reduce(org.apache.hadoop.io.Text key,
          Iterator<CrawlDatum> values,
          org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,CrawlDatum> output,
          org.apache.hadoop.mapred.Reporter reporter)
            throws IOException
  - Specified by: 
  - <code>reduce</code> in interface <code>org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.text,crawldatum,org.apache.hadoop.io.text,crawldatum></org.apache.hadoop.io.text,crawldatum,org.apache.hadoop.io.text,crawldatum></code> 
  - Throws: 
  - <code>IOException</code>

Copyright © 2014 The Apache Software Foundation