HDFS—NameNode包

org.apache.hadoop.hdfs.server.namenode

版本 hadoop  release1.0.0

BlockMaps.java
This class maintains the map from a block to its metadata. block’s metadata currently includes INode it belongs to and the datanodes that store the block.
维护了块到块元数据的映射,块元数据包括这个块属于哪个INode 以及哪个数据服务器保存了这个块。

CancelDelegationTokenServlet.java
Cancel delegation tokens over http for use in hftp.

CheckpointSignature.java
A unique signature intended to identify checkpoint transactions.
确定检查点交互的独特签名

ContentSummaryServlet.java
Servlets for file checksum
文件校验码的小程序

CorruptReplicasMap.java
Stores information about all corrupt blocks in the File System.A Block is considered corrupt only if all of its replicas are corrupt. While reporting replicas of a Block, we hide any corrupt copies. These copies are removed once Block is found to have expected number of good replicas. Mapping: Block -> TreeSet<DatanodeDescriptor>
存储文件系统中所有崩溃块的信息。只有块的所有备份都损坏了才认为块损坏了。在报告块的副本时隐藏了损坏副本。当已经有预期数量的副本时,则删掉这些块。

DatanodeDescriptor.java
DatanodeDescriptor tracks stats on a given DataNode, such as available storage capacity, last update time, etc., and maintains a set of blocks stored on the datanode. This data structure is a data structure that is internal to the namenode. It is *not* sent over-the-wire to the Client or the Datnodes. Neither is it stored persistently in the fsImage.
DatanodeDescriptor跟踪指定DataNode 的状态,比如可用的存储空间,最后一次更新时间等,并维护DataNode 上的块。这是一个对内的数据结构,并不发送给Client 和其他DataNodes,也不是永久的存在fsImage中。

DecommissionManager.java
Manage node decommissioning
管理节点的失效

DfsServlet.java
A base class for the servlets in DFS.

EditLogInputStream.java
A generic abstract class to support reading edits log data from persistent storage. It should stream bytes from the storage exactly as they were written into the #{@link EditLogOutputStream}.
通用抽象类支持从永久性存储中读取edits log 数据。流式读取和写入一样

EditLogOutputStream.java
A generic abstract class to support journaling of edits logs into a persistent storage.
通用抽象类支持将edits log写入永久性存储。

FileChecksumServlets.java
Servlets for file checksum
检查文件校验的小程序

FileDataServlet.java
Redirect queries about the hosted filesystem to an appropriate datanode. @see org.apache.hadoop.hdfs.HftpFileSystem

FsckServlet.java
This class is used in Namesystem’s web server to do fsck on namenode.

FSDirectory.java
FSDirectory stores the filesystem directory state. It handles writing/loading values to disk, and logging changes as we go. It keeps the filename->blockset mapping always-current and logged to disk.
FSDirectory保存文件系统目录状态,处理写、装载目录内容到磁盘,并记录发生的变化。它保存了文件名到对应文件的块集合的映射并记录到磁盘。

FSEditLog.java
FSEditLog maintains a log of the namespace modifications.
维持了名字空间的修改。

FSImage.java
FSImage handles checkpointing and logging of the namespace edits.
处理检查点和记录名字空间编辑信息

FSNamesystem.java
FSNamesystem does the actual bookkeeping work for the DataNode. It tracks several important tables.
1)  valid fsname –> blocklist  (kept on disk, logged)
2)  Set of all valid blocks (inverted #1)
3)  block –> machinelist (kept in memory, rebuilt dynamically from reports)
4)  machine –> blocklist (inverted #2)
5)  LRU cache of updated-heartbeat machines
FSNamesystem 为DataNode 增添进行记录,并维护以下几张表:
1) 可用文件名–》块列表(记录在硬盘上)
2) 所有可用的块(#1 的转置)
3) 块–》机器列表(内存保存,从DataNode的报告中动态的重建)
4) 机器–》块列表(#3 的转置)
5) LRU 缓存最近更新心跳信息的机器

FSPermissionChecker.java
Perform permission checking in {@link FSNamesystem}.
在FSNamesystem 中执行权限检查

GetDelegationTokenServlet.java
Serve delegation tokens over http for use in hftp.

GetImageServlet.java
This class is used in Namesystem’s jetty to retrieve a file. Typically used by the Secondary NameNode to retrieve image and edit file for periodic checkpointing.
在Namesystem 中取出一个文件,通常是用于Secondary NameNode 取得一个image 和edit file 用于阶段性的检查点。

Host2NodesMap.java
null

INode.java
We keep an in-memory representation of the file/block hierarchy. This is a base INode class containing common fields for file and directory inodes.
我们保存在内存中代表着文件、块的层次关系。这个基础INode 类为文件和目录inodes 保存着通用的一些字段。

INodeDirectory.java
Directory INode class.
目录INode 类

INodeDirectoryWithQuota.java
Directory INode class that has a quota restriction
有容量限制的目录INode 类

INodeFile.java
File INode class.
文件INode 类

INodeFileUnderConstruction.java
File INode class that has constructions.
有限制的文件INode 类

JspHelper.java
null

LeaseExpiredException.java
The lease that was being used to create this file has expired.
用于创建这个文件的租约到期的文件

LeaseManager.java
LeaseManager does the lease housekeeping for writing on files. This class also provides useful static methods for lease recovery.
LeaseManager用于将租约记录到文件中,这个类也提供了可用的静态方法用于租约恢复

ListPathsServlet.java
Obtain meta-information about a filesystem. @see org.apache.hadoop.hdfs.HftpFileSystem
获取文件系统的元数据

NameCache.java
Caches frequently used names to facilitate reuse.
缓存常用的名字有助于重用

NameNode.java
NameNode serves as both directory namespace manager and “inode table” for the Hadoop DFS.  There is a single NameNode running in any DFS deployment.  (Well, except when there is a second backup/failover NameNode.)
NameNode既是目录名字空间管理者,也是Hadoop 分布式文件系统的“inode table”。在每个hadoop 分布式文件系统的部署中只有一个NameNode。

NamenodeFsck.java
This class provides rudimentary checking of DFS volumes for errors and sub-optimal conditions.
为错误和非最佳状态下进行初步检查

NameNodeMXBean.java
This is the JMX management interface for namenode information
NameNode的JMX java 管理扩展接口

NotReplicatedYetException.java
The file has not finished being written to enough datanodes yet.

PendingReplicationBlocks.java
PendingReplicationBlocks does the bookkeeping of all blocks that are getting replicated.
记录所有的块都被复制

PermissionChecker.java
Perform permission checking in {@link FSNamesystem}.
在FSNamesystem 中执行权限检查

RenewDelegationTokenServlet.java
Renew delegation tokens over http for use in hftp.

ReplicationTargetChooser.java
The class is responsible for choosing the desired number of targets for placing block replicas. The replica placement strategy is that if the writer is on a datanode, the 1st replica is placed on the local machine, otherwise a random datanode. The 2nd replica is placed on a datanode that is on a different rack. The 3rd replica is placed on a datanode which is on the same rack as the first replca.
这个类负责选择所需数量的目标机器用于放置块副本。副本放置策略是如果writer是在一个数据节点上,则第一个副本放于本机,如果writer不在数据节点上,则随机选取个数据节点。第二个副本放置在一个不同的机架上。第三个副本放置在和第一个副本相同的机架上。

SafeModeException.java
This exception is thrown when the name node is in safe mode. Client cannot modified namespace until the safe mode is off.
当NameNode在安全模式下抛出这个异常,安全模式退出前客户端都不能够修改名字空间

SecondaryNameNode.java
The Secondary NameNode is a helper to the primary NameNode. The Secondary is responsible for supporting periodic checkpoints of the HDFS metadata. The current design allows only one Secondary NameNode per HDFs cluster. The Secondary NameNode is a daemon that periodically wakes up (determined by the schedule specified in the configuration), triggers a periodic checkpoint and then goes back to sleep. The Secondary NameNode uses the ClientProtocol to talk to the primary NameNode.
Secondary NameNode 用于协助primary NameNode。用于阶段性的建立HDFS 元数据的检查点。现在的设计还仅允许在每个HDFS 集群中只有一个Secondary NameNode。Secondary NameNode是一个守护进程,定期地醒来并触发一个周期性的检查点然后又去睡觉了。Secondary NameNode 利用ClientProtocol 与primary NameNode 进行交互。

SerialNumberManager.java
Manage name-to-serial-number maps for users and groups.
管理名字到序列码的映射

StreamFile.java
null

TransferFsImage.java
This class provides fetching a specified file from the NameNode.
从NameNode 中取指定文件

UnderReplicatedBlocks.java
Class for keeping track of under replication blocks Blocks have replication priority, with priority 0 indicating the highest Blocks have only one replicas has the highest
块复制的优先级问题,优先级0 具有最高优先级,比如只有一个副本的块就具有最高优先级。

UpgradeManagerNamenode.java
Upgrade manager for name-nodes.
升级NameNodes

UpgradeObjectNamenode.java
Base class for name-node upgrade objects. Data-node upgrades are run in separate threads.
升级NameNodes

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据