2016年5月 – VOID001's WOWO

[Linux C]ELF Trick — 获取程序内的函数名及地址

五月 30, 2016 VOID001 Comments 0 Comment

这一切的起源来自于一个群的对话:

B 23:27:38
我定义了一个 C 的函数
abc_hello_world()
然后我想通过命令行通过参数
xxxxx abc hello world
把三个参数，自动作为函数名给弄进去
然后输出那个函数的执行结果
在 C 里怎么搞啊

首先我们的第一想法都是 “宏”

网络部 R 籓绽23:29:13
我记得宏定义有一种方式可以拼接字符串吧
网络部 R 籓绽23:29:25
我没弄过窝想想看
A 23:30:05
好像没办法实现这个，编译之后都是二进制了，函数名在优化过程中都没了
A 23:30:26
我试试宏
网络部 R 籓绽23:30:29
宏定义的确可以拼接字符串的说
网络部 R 籓绽 23:30:32
你试试看

然后, 这些抽风的程序员们感觉, 宏这种用法太无趣了(笑) 于是就有了下面这段对话

网络部 R 籓绽23:37:30
知道这个原理我们c就可以实现了吧
A 23:38:09
他是解释器，变量名会一直保存，但是 C 编译后就没有变量名了
网络部 R 籓绽 23:38:47
有的 c可以留符号表的
A 23:39:03
(⊙o⊙)…怎么做
网络部 R 籓绽 23:39:04
我们可不可以查符号表找地址呢
网络部 R 籓绽 23:39:12
调试信息呀
B 23:39:15
查符号表地址那不就是函数指针了？
网络部 R 籓绽23:39:21
恩
网络部 R 籓绽23:39:31
先查函数名然后
网络部 R 籓绽 23:39:36
找到函数的地址
网络部 R 籓绽 23:39:39
然后调用
A 23:39:51
(⊙o⊙)…阔以耶。。。

然后, 就如上文所说, 准备查symtab中的信息, 然后得到函数名以及对应的函数指针入口地址, 于是就有了下面这段代码

[因为今天考试, 代码的解释, 原理解释什么的, 等儿童节后再写哇卡卡卡]

先看程序运行结果

我们这个程序内有func1 func2 func3 extern_fun main 这么几个函数, 我们来看一下运行结果

我们可以在程序内任意添加新的函数, 而不需要修改其他部分的代码, 重新编译之后, 即可通过输入那个函数的名称,调用相应的函数. 同时也可以编译链接多个源文件 (extern_fun 就是在其他文件内定义的函数) 并且调用到他们

下面是具体的实现代码:

/*************************************************************************
    > File Name: a.c
    > Author: VOID_133
    > ################### 
    > Mail: ################### 
    > Created Time: Mon 30 May 2016 07:53:59 AM CST
 ************************************************************************/
#include<stdio.h>
#include<stdlib.h>
#include<elf.h>
#include<libelf.h>
#include<unistd.h>
#include<fcntl.h>
#include<gelf.h>
#include<string.h>

//Define a seris of function for get ELF Data

typedef struct fun_tab {
    char name[1000];
    int addr;
    struct fun_tab *next;
}FTable;

void extern_fun();

FTable *get_fun_table(char *progname)
{
    int fd, count;
    size_t entries;
    Elf *elf = NULL;
    Elf_Scn *elf_section = NULL;
    Elf_Data *elf_data = NULL;
    Elf_Arsym *elf_arsym = NULL;
    GElf_Shdr shdr;

    fd = open(progname, O_RDONLY);
    if(fd < 0)
    {
        perror("open");
        exit(1);
    }
    // Call elf_version before all elf operations
    if(elf_version(EV_CURRENT) == EV_NONE)
    {
        perror("elf_version");
        exit(1);
    }
    elf = elf_begin(fd, ELF_C_READ, (Elf *) NULL);
    if(elf == NULL)
    {
        perror("elf");
        exit(1);
    }
    // Get the first section of ELF File
    elf_section = elf_getscn(elf, 0);

    while((elf_section = elf_nextscn(elf, elf_section)) && elf_section != NULL)
    {
        // Read the data in the section
        gelf_getshdr(elf_section, &shdr);
        if(shdr.sh_type == SHT_SYMTAB)
            break;
    }
    elf_data = elf_getdata(elf_section, NULL);
    count = shdr.sh_size;

    //Print the symbol Table
    FTable *head = (FTable *)malloc(sizeof(FTable));
    FTable *ptr = head;
    for(int ii = 0; ii < count; ii++)
    {
        GElf_Sym sym;
        gelf_getsym(elf_data, ii, &sym);
        char *sym_name = elf_strptr(elf, shdr.sh_link, sym.st_name);
        if(sym_name[0] != '_' && sym_name != NULL && sym_name[0] != ' ' && sym.st_value != 0)
        {
            //printf("%s = %X\n", elf_strptr(elf, shdr.sh_link, sym.st_name), sym.st_value);
            strcpy(ptr->name, sym_name);
            ptr->addr = sym.st_value;
            ptr->next = (FTable *)malloc(sizeof(FTable));
            ptr = ptr->next;
        }
    }
    elf_end(elf);
    close(fd);
    return head;
}

void print_fun_table(FTable *head)
{
    FTable *ptr = head;
    while(ptr != NULL)
    {
        printf("%s [%X]\n", ptr->name, ptr->addr);
        ptr = ptr->next;
    }
    return;
}

void exec_fun_by_name(FTable *head, char *funname)
{
    FTable *ptr = head;
    while(ptr != NULL)
    {
        if(!strcmp(funname, ptr->name))
        {
            //Use the function
            printf("Found\n");
            void (*f)();
            f = (void (*)())ptr->addr;
            f();
            return ;
        }
        ptr = ptr->next;
    }
    if(ptr == NULL)
    {
        printf("Error: Cannot find the function named %s\n", funname);
    }
}

void func1()
{
    printf("I am fun1, ~~~ ASALSJAKLJSA\n");
}

void func2()
{
    printf("This is fun2 |~~~~~dsakld QWQ\n");
}

void func3()
{
    // I am fun 3
}

int main(int argc, char *argv[])
{
    char *progname = (char *)malloc(sizeof(argv[0]));
    progname = argv[0];
    if(progname == NULL)
    {
        perror("malloc");
        exit(1);
    }
    FTable *tab = get_fun_table(progname);

    printf("Input the function name you want to call\n");
    char funname[1000];

    while(1)
    {
        printf(">");
        scanf("%s", funname);
        exec_fun_by_name(tab, funname);
    }
    return 0;
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

/*************************************************************************

> File Name: a.c

> Author: VOID_133

> ###################

> Mail: ###################

> Created Time: Mon 30 May 2016 07:53:59 AM CST

************************************************************************/

#include<stdio.h>

#include<stdlib.h>

#include<elf.h>

#include<libelf.h>

#include<unistd.h>

#include<fcntl.h>

#include<gelf.h>

#include<string.h>

//Define a seris of function for get ELF Data

typedef struct fun_tab {

char name[1000];

int addr;

struct fun_tab *next;

}FTable;

void extern_fun();

FTable *get_fun_table(char *progname)

{

int fd, count;

size_t entries;

Elf *elf = NULL;

Elf_Scn *elf_section = NULL;

Elf_Data *elf_data = NULL;

Elf_Arsym *elf_arsym = NULL;

GElf_Shdr shdr;

fd = open(progname, O_RDONLY);

if(fd < 0)

{

perror("open");

exit(1);

}

// Call elf_version before all elf operations

if(elf_version(EV_CURRENT) == EV_NONE)

{

perror("elf_version");

exit(1);

}

elf = elf_begin(fd, ELF_C_READ, (Elf *) NULL);

if(elf == NULL)

{

perror("elf");

exit(1);

}

// Get the first section of ELF File

elf_section = elf_getscn(elf, 0);

while((elf_section = elf_nextscn(elf, elf_section)) && elf_section != NULL)

{

// Read the data in the section

gelf_getshdr(elf_section, &shdr);

if(shdr.sh_type == SHT_SYMTAB)

break;

}

elf_data = elf_getdata(elf_section, NULL);

count = shdr.sh_size;

//Print the symbol Table

FTable *head = (FTable *)malloc(sizeof(FTable));

FTable *ptr = head;

for(int ii = 0; ii < count; ii++)

{

GElf_Sym sym;

gelf_getsym(elf_data, ii, &sym);

char *sym_name = elf_strptr(elf, shdr.sh_link, sym.st_name);

if(sym_name[0] != '_' && sym_name != NULL && sym_name[0] != ' ' && sym.st_value != 0)

{

//printf("%s = %X\n", elf_strptr(elf, shdr.sh_link, sym.st_name), sym.st_value);

strcpy(ptr->name, sym_name);

ptr->addr = sym.st_value;

ptr->next = (FTable *)malloc(sizeof(FTable));

ptr = ptr->next;

}

elf_end(elf);

close(fd);

return head;

}

void print_fun_table(FTable *head)

{

FTable *ptr = head;

while(ptr != NULL)

{

printf("%s [%X]\n", ptr->name, ptr->addr);

ptr = ptr->next;

}

return;

}

void exec_fun_by_name(FTable *head, char *funname)

{

FTable *ptr = head;

while(ptr != NULL)

{

if(!strcmp(funname, ptr->name))

{

//Use the function

printf("Found\n");

void (*f)();

f = (void (*)())ptr->addr;

f();

return ;

}

ptr = ptr->next;

}

if(ptr == NULL)

{

printf("Error: Cannot find the function named %s\n", funname);

}

void func1()

{

printf("I am fun1, ~~~ ASALSJAKLJSA\n");

}

void func2()

{

printf("This is fun2 |~~~~~dsakld QWQ\n");

}

void func3()

{

// I am fun 3

}

int main(int argc, char *argv[])

{

char *progname = (char *)malloc(sizeof(argv[0]));

progname = argv[0];

if(progname == NULL)

{

perror("malloc");

exit(1);

}

FTable *tab = get_fun_table(progname);

printf("Input the function name you want to call\n");

char funname[1000];

while(1)

{

printf(">");

scanf("%s", funname);

exec_fun_by_name(tab, funname);

}

return 0;

}

目前只能调用返回值 ,参数列表为空的函数, 改进的话, 可以实现多种参数,多种返回值的调用

NEUOJ Tech Talk #1

五月 29, 2016 VOID001 Comments 0 Comment

开始之前

First Slide , Wait till 18:30

欢迎大家来到NEUOJ Tech Talk技术交流会，今天，我们会将在开发东北大学学生在线测评系统(NEUOJ)中使用的种种技术，以通俗易懂的方式，分享给热爱技术的你们，那么在我们的Talk开始之前，让我们首先感谢那些对我们OJ开发给出种种建议的各位同学，并颁发我们的奖品

颁奖环节

我们有请吴老师，为他们进行颁奖

感谢吴老师，我们的NEUOJ 持续提供给大家进行渗透测试，安全测试的平台，并且对成功找到并且利用我们OJ的漏洞的同学，颁发丰厚的奖品，从32GU盘到 2T移动硬盘，视漏洞大小而定，对为OJ提出宝贵建议，报出非常有价值的bug的同学，我们会进行奖励欢迎大家踊跃参与

那么现在，就让我们开始吧～

[Slide 2]

Tech Talk Starts

首先，我们来了解一下，NEUOJ到底是什么

我们会先介绍什么是Online Judge ，然后介绍为什么我们要开发NEUOJ， NEUOJ的特点是什么，然后会给大家展示NEUOJ的开发史

[Slide 3]

什么是OJ呢， OJ是一个可以在线编译提交的代码，并且自动对代码的运行结果进行判断，并且返回给用户判断之后的结果这样一个系统， O 代表Online, J代表 Judge。我们下面来展示一下，什么是在线编译器。 [切换图片]大家可以看到，这是一个在线编译网站 codepad 的界面，我们写了一个Hello world …, 然后选择我们的编程语言，C语言，大家可以看到，这里支持C/C++/Haskell/PHP等多种编程语言的在线编译运行 [对这个图片进行解释]，然后当我们点击提交之后，[切换图片] 就会看到这个界面: 我们可以看到，这里已经输出了我们程序的运行结果。这就是Online Compiler, 而Online Compiler 的功能的基础上，再加上自动测评(Judge)的功能，就是我们的Online Judge.[图片展示] [简单介绍]

[Slide 4]

现在流行的Online Judge 平台有北京大学的POJ, 杭州电子科技大学的HDUOJ, BestCoder, 以及国外知名的Leetcode, Codeforces, 还有Topcoder，下面，我们来看看NEUOJ是什么

我们的NEUOJ使用的是当今最流行的Web框架 Laravel， Laravel框架具有极高的可扩展性，丰富的插件系统，以及很好的安全性。同時，我们的测评后端具有高度的扩展性，可以支持各种方式的测评，目前我们只支持了最基本的测评，之后，我们会考虑增加更有趣的功能，如人机对弈测评机制等测评方式，之后，我们会详细介绍我们是如何实现的高度扩展性的测评系统 . 我们的NEUOJ不同于codeforces是为了举行各种比赛而建立的OJ，NEUOJ的服务是提供给广大学生，我们的天梯训练training系统提供给大家一个循序渐进提升自己代码能力的路线，用于让我校的各个专业的对编程有兴趣的同学提高自己的编程能力，也给老师们进行计算机相应教学，提供一个测评编程能力，算法能力的平台。同时也为那些想要通过技术甄选成员的组织，以及想要带学生做项目的老师，提供一个测评能力的平台，可以通过OJ提供的排行榜ranklist，以及大家的代码统计信息，选出能力出色的同学加入

[Slide 5]

下面我们来一起看一下NEUOJ的开发历史:

2015-11-25 在我们开完第一次OJ开发会议之后，进行了第一次的代码提交，截止目前，我们OJ已经有348个提交， 20000+行代码，近100个公开Issue，[一张张图片展示] [并进行解释]

我们的OJ目前已经支持的主要功能有，题目管理，竞赛系统，天梯训练系统，预计2.0版本开发完毕，要支持的功能有:[切图]

DEMO 下面我们请一位同学上台体验一下，我们的OJ的管理员后台，实现添加一道题目，并且添加一个比赛

[Slide 6]

那么对于这样的一个NEUOJ，我们是如何构建的呢，下面我们会从，如何对代码进行管理，我们使用的Web框架，以及NEUOJ的核心，测评端，以及NEUOJ的界面设计，前端技术几个方面进行介绍

[Slide 7]

首先，让我们来介绍NEUOJ的代码管理系统～ git

对于348commits 6 — 8 developers 合作的一个项目，我们是如何实现多人协作开发，即多人同时修改同一份代码的？

答案就是 GIT

相信在座的各位同学应该有日常使用git的，那么什么是git呢？谁能来回答一下？

那么下面让我们来了解一下，git到底是什么

我们设想这样一个情景 [这里举例 git的版本管理]

下面我们请NEUOJ后端开发组的洋葱同学，结合刚刚我们说明的情形，为大家演示一下git的使用

[Slide 8]

结合图片讲MVC

介绍Eloquent ORM

[Slide 9]

结合图片介绍Judgehost

简单介绍RESTfulAPI

简单介绍如何保护我们的运行环境

[Slide 10]

下面请NEUOJ的前端负责人 su123456 同学介绍一下， OJ前端开发所用到的技术

一个项目，最重要的并不是编写代码进行开发，而是对项目进行维护，管理，保证项目的健壮性，可用性，以及优化项目的性能，接下来就来了解一下，我们是如何保证NEUOJ的质量的

我们将介绍如何提升OJ的性能，以及如何自动化的测试代码的质量，以及保证我们的服务器的安全性三个方面来介绍

[Slide 11]

[Slide 12]

[Slide 13]

安全DEMO 暴力破解密码

在这次Talk的最后，我想要介绍一下，这两个词语 Opensource 以及 Geek [jieshao]

最后，感谢大家参与这次技术交流会，也希望我们今天所讲的能给大家一定的帮助，同时欢迎更多的同学研究技术，学习技术，也欢迎对OJ开发感兴趣的同学加入我们，同时，欢迎大家加入我们的NEUOJ Geek 群，大家一起分享技术，交流技术。NEUOJ　Tech Talk #1就到这里了，谢谢大家～

[Linux C内核] 信号

五月 27, 2016 VOID001 Comments 0 Comment

下面对Linux 下的信号的产生, 响应机制, 以及可重入(reentrant)进行介绍~

[首先再次给网易云音乐点个赞~\(≧▽≦)/~, 在写博客的时候听着歌而且还有歌词滚动, 真的太棒了 ~~~~]

[不行, 我要附上一张截图]

Netease-Blog

恩,心满意足的窝开始写文章了 ~~

什么是信号

在Interprocess Communication(IPC)的情形中,如何让不同的process进行通信是一个很重要的需要解决的问题, 而信号就是不同的进程之间通信的一种方式, 而在Linux下运行的每个程序, 都会受到内核的控制, 内核就通过信号告知进程一些控制信息, 如中断执行, 停止执行, 进程和进程之间也会进行信号的传递, 比如父进程fork出的子进程, 子进程退出的时候会发送SIGCHLD信号给父进程, 告诉父进程, 我运行完啦~, 那么Linux是如何实现信号的捕捉,并且送达到相应的程序的呢? 下面我们举个具体例子来看一下

我们以最简单的情景: 用户在terminal下执行一个程序,然后由用户通过Ctrl+C打断程序的执行,返回到terminal下的整个过程中, 信号的流向进行说明

输入命令运行程序, 在Shell下启动一个前台进程, 分配了相应的PCB
程序运行过程中,用户按下 Ctrl+C, 因为有键盘输入的介入, 产生了硬件中断, CPU从用户态切换到内核态,去处理中断
终端的相应驱动程序,将Ctrl+C解释为 SIGINT信号, 并记录在当前运行在终端中的程序的PCB中
当要从内核态切换回用户态之前, 会对PCB中记录的信号, 进行处理, 检察到PCB中有 SIGINT信号, 于是对信号进行处理, 而目前这个信号的处理动作是终止进程执行, 于是这个程序终止了

信号分为实时信号和非实时信号, kill 指令后面可以接信号常量, 根据信号常量对应的数值区分实时/非实时信号 > 34的是实时信号我们这里也值讨论 < 34的信号(即非实时的信号) 关于信号的更详细介绍, 参考 signal(7)即可

产生信号的条件

上面只说了通过Ctrl+C产生一个信号发送给进程, 那么还有什么方式产生一个信号呢? 下面列出主要的几种方式

用户在终端进行某些按键的时候, 终端驱动程序 会发送信号给相应的进程
硬件异常也会产生信号, 如当前进程执行了除以0指令, CPU的运算单元会产生异常
一个进程调用 kill 系统函数可以发送信号给某个进程, 注意这里的kill不仅仅能够kill掉某个进程, kill可以用来发送各种信号给其他的进程
当内核检测到某种软条件可以通过信号通知进程, 常见的是alarm产生的SIGALRM信号 & 管道通信产生的SIGPIPE信号

下面我们通过一个alarm 小程序查看一下产生信号的条件之一软条件由内核产生信号:

/*************************************************************************
    > File Name: sigalarm.c
    > Author: VOID_133
    > ################### 
    > Mail: ################### 
    > Created Time: Tue 24 May 2016 04:44:22 PM CST
 ************************************************************************/
#include<unistd.h>
#include<stdio.h>

int main(void)
{
    alarm(1);
    int cnt = 0;
    while(1)
    {
        printf("%d\n", cnt++);
    }
    return 0;
}

/*************************************************************************

> File Name: sigalarm.c

> Author: VOID_133

> ###################

> Mail: ###################

> Created Time: Tue 24 May 2016 04:44:22 PM CST

************************************************************************/

#include<unistd.h>

#include<stdio.h>

int main(void)

{

alarm(1);

int cnt = 0;

while(1)

{

printf("%d\n", cnt++);

}

return 0;

}

执行该程序, 该程序会在1s(精确到 0.01s) 时间内数数, 然后时间到了就会退出, 我们用 time 运行一下看看

$time ./sigalarm

1	$time ./sigalarm

结果如下

61102
61103
[2]    3215 alarm      ./sigalarm
./sigalarm  0.03s user 0.07s system 9% cpu 1.001 total

61102

61103

[2] 3215 alarm ./sigalarm

./sigalarm 0.03s user 0.07s system 9% cpu 1.001 total

可以看出来这里就是产生了SIGALRM信号, 然后该进程接收到这个信号之后, 默认的处理动作就是退出程序的执行, 因而程序退出了, 之后我们会介绍如何更改默认的信号处理方式, 先不要着急~~

信号的本质

如果想要对信号进行更高级的操作的话, 那么我们需要了解信号的本质, 也就是信号在内核中到底是如何存储的, 如何表示的, 我们用一张图来说明信号在内核中的表示

Document_2016-05-27_20-24-40

黄色的是PCB, 紫色的就是PCB中负责存储信号相关信息的部分, 然后我们来看一下这个部分, 这个部分规定了该进程对每个信号的处理, 响应方式, pending位表示信号是否已经到达(叫做产生更准确,不过我认为到达更形象), block位表示是否屏蔽这个信号的响应(即这个信号到来之后,将信号阻塞住)WARN:”阻塞”和”忽略这个信号” 不是一个概念, “忽略这个信号” 是在信号递达之后的一个对信号的响应方式, 而阻塞, 这是不响应这个信号, 就好比你和你女票闹矛盾了, 你为了防止他不接电话,特地换了个新手机卡,给她打电话,结果电话关机(我们假设她真的是关机)[这是阻塞], 然后, 你等到晚上, 终于电话开机了, 然后你打过去, 她接起来,你开始哄她, 然而整个过程你女票一句话没说,并且淡定的挂了电话, [忽略]了你的信号, 是不是很生动且形象desu~~(怕被广大单身程序员打死的修罗已经逃走)

Hmm, 我们还是看上面那个图, 那个图上还有一部分我们没有说明, 就是 handler部分, 通过你工资就能看出来,这里指定了对每个信号的处理方式对一个信号的处理方式有以下几种:

默认处理方式 SIG_DFL 这个就是默认的处理方式啦,没什么解释的
SIG_IGN 忽略这个信号
handler_function 用户自己定义的handler函数~ 这种方式又叫做捕捉一个信号

相信通过上面的内容,大家对信号的产生, 递达 , 处理有一个基本的了解了~~ 接下来我们就用一个具体的例子演示一下上述内容

一个无法通过Ctrl + C退出的程序

下面先演示一个最简单的例子, 然后我们还会给出他的升级版, 一个除了****都怎样都无法退出的程序, 首先是第一个程序

/*************************************************************************
    > File Name: sigset_1.c
    > Author: VOID_133
    > ################### 
    > Mail: ################### 
    > Created Time: Thu 26 May 2016 07:32:22 AM CST
 ************************************************************************/
#include<stdio.h>
#include<stdlib.h>
#include<unistd.h>
#include<signal.h>

void printsigset(const sigset_t *p)
{
    int i = 0;
    for(i = 1; i < 32; i++)
    {
        if(sigismember(p, i) == 1)
            putchar('1');
        else
            putchar('0');
    }
    puts("");
}

int main(void)
{  
    sigset_t set, p;
    //Init the signal
    sigemptyset(&set);
    //Add signals
    alarm(5);
    if(sigaddset(&set, SIGINT) < 0)
    {
        perror("add SIGKILL err");
        exit(1);
    }
    sigprocmask(SIG_BLOCK, &set, NULL);
    while(1)
    {
        sigpending(&p);
        printsigset(&p);
        sleep(1);
    }
}

/*************************************************************************

> File Name: sigset_1.c

> Author: VOID_133

> ###################

> Mail: ###################

> Created Time: Thu 26 May 2016 07:32:22 AM CST

************************************************************************/

#include<stdio.h>

#include<stdlib.h>

#include<unistd.h>

#include<signal.h>

void printsigset(const sigset_t *p)

{

int i = 0;

for(i = 1; i < 32; i++)

{

if(sigismember(p, i) == 1)

putchar('1');

else

putchar('0');

}

puts("");

}

int main(void)

{

sigset_t set, p;

//Init the signal

sigemptyset(&set);

//Add signals

alarm(5);

if(sigaddset(&set, SIGINT) < 0)

{

perror("add SIGKILL err");

exit(1);

}

sigprocmask(SIG_BLOCK, &set, NULL);

while(1)

{

sigpending(&p);

printsigset(&p);

sleep(1);

}

这个程序运行后, 5秒钟之后才会退出, 之间无论你怎么按Ctrl+C都没法停下这个程序, 不过你会发现按完Ctrl+C之后, 输出的东西改变了, 有一位变成了1 , 这个稍后会进行解释

我们来解释一下这段代码, 先看main函数首先我们用sigempty将信号集 set 清空这里解释一下信号集:

信号集就是一个用于存储所有信号的信息的数据集合, 每一个信号的”有效/无效” 都用一个bit来进行表示, 至于具体哪个bit表示哪个信号,这个是不确定的, 由操作系统来决定, 所以对其的操作都要通过相应的函数来实现,而不能直接修改某个bit.

这里就是将所有的信号都设置为”无效” ,不过这里还没有进行任何操作, 要对信号进行操作的时候, 还要用其他的函数,将set作为参数传递进去. 这里可以将sigemptyset理解为不选择任何一个信号进行操作, 和这个函数相对的是 sigfillset 意为选中所有信号进行操作

然后我们将SIGINT信号选中, 使用 sigaddset, 他的兄弟函数是 sigdelset

然后将选中的信号进行操作 sigprocmask, 设定屏蔽字, 让当前进程屏蔽 SIGINT, 即阻塞该信号

另外sigpending函数是取出现在产生但是还未递达的信号, 生成一个信号集, 存储它的信息, 然后使用自定义的一个函数打印出来了这个信号集的内容, 现在我们可以解释为什么按下Ctrl+C之后, 有一位变成了 1, 因为这个表示SIGINT, 信号的产生

然后我们来看下面这个加强版的, “任何”信号都杀不死的程序

/*************************************************************************
    > File Name: sigset_1.c
    > Author: VOID_133
    > ################### 
    > Mail: ################### 
    > Created Time: Thu 26 May 2016 07:32:22 AM CST
 ************************************************************************/
#include<stdio.h>
#include<stdlib.h>
#include<unistd.h>
#include<signal.h>

void printsigset(const sigset_t *p)
{
    int i = 0;
    for(i = 1; i < 32; i++)
    {
        if(sigismember(p, i) == 1)
            putchar('1');
        else
            putchar('0');
    }
    puts("");
}

int main(void)
{
    printf("This program will run forever, you cannot stop it\n");
    printf("If you are sure to run it, press enter, else Ctrl + C NOW");
    getchar();
    sigset_t set, p;
    //Init the signal
    sigfillset(&set);
    //Add signals
    alarm(5);
    if(sigaddset(&set, SIGKILL) < 0)
    {
        perror("add SIGKILL err");
        exit(1);
    }
    sigprocmask(SIG_BLOCK, &set, NULL);
    while(1)
    {
        sigpending(&p);
        printsigset(&p);
        sleep(1);
    }
}

/*************************************************************************

> File Name: sigset_1.c

> Author: VOID_133

> ###################

> Mail: ###################

> Created Time: Thu 26 May 2016 07:32:22 AM CST

************************************************************************/

#include<stdio.h>

#include<stdlib.h>

#include<unistd.h>

#include<signal.h>

void printsigset(const sigset_t *p)

{

int i = 0;

for(i = 1; i < 32; i++)

{

if(sigismember(p, i) == 1)

putchar('1');

else

putchar('0');

}

puts("");

}

int main(void)

{

printf("This program will run forever, you cannot stop it\n");

printf("If you are sure to run it, press enter, else Ctrl + C NOW");

getchar();

sigset_t set, p;

//Init the signal

sigfillset(&set);

//Add signals

alarm(5);

if(sigaddset(&set, SIGKILL) < 0)

{

perror("add SIGKILL err");

exit(1);

}

sigprocmask(SIG_BLOCK, &set, NULL);

while(1)

{

sigpending(&p);

printsigset(&p);

sleep(1);

}

这个代码不用再解释了, 我们直接暴力的屏蔽了所有的信号当然什么信号都没有用啦~~(可怕

这个程序运行起来之后, 只有一种方式不重启电脑停止这个程序: 那就是向该进程发送SIGKILL信号, 引用兔子的一句话, “没什么是一发SIGKILL不能杀死的, 如果有的话,就两发”. 可是SIGKILL为什么能够不受我们屏蔽字的影响, 而使程序退出呢? 注意我还特意对SIGKILL信号位选中, 生怕是因为我没选中才导致这个信号没被屏蔽.

The SIGKILL signal is sent to a process to cause it to terminate immediately (kill). In contrast to SIGTERM and SIGINT, this signal cannot be caught or ignored, and the receiving process cannot perform any clean-up upon receiving this signal.

根据wiki的解释, SIGKILL这个信号根本就没有被送到当前进程,当前进程没有接收到这个信号, 直接被Kernel杀死了(好可怜, 不明不白的就挂了) en, 这样就能解释同上面的问题了

信号捕捉的实现自定义信号处理函数

下面我们通过一段代码实现一个自定义的闹钟demo, 当时间到了的时候, 会显示timeup并且退出, 期间可以通过Ctrl-C打断,打断后会输出一个萌萌的表情说明,然后退出, 代码如下

/*************************************************************************
    > File Name: sigcatch.c
    > Author: VOID_133
    > ################### 
    > Mail: ################### 
    > Created Time: Thu 26 May 2016 09:03:32 AM CST
 ************************************************************************/
#include<stdio.h>
#include<signal.h>
#include<unistd.h>
#include<stdlib.h>

typedef struct sigaction Action;
void interrupt();
void alarmclock();

int main(void)
{
    sigset_t set;
    sigemptyset(&set);
    sigaddset(&set, SIGALRM);
    Action sigact;
    sigact.sa_mask = set;
    sigact.sa_handler = alarmclock;
    sigaction(SIGALRM, &sigact, NULL);

    sigemptyset(&set);
    sigaddset(&set, SIGINT);
    sigact.sa_mask = set;
    sigact.sa_handler = interrupt;
    sigaction(SIGINT, &sigact, NULL);
    //This is useless, non-realtime signal will be sent only once
    printf("The program will sleep for 3 secs...\n");
    while(1)
    {
        alarm(3);
        pause();
    }
    return 0;
}

void interrupt()
{
    printf("0.0 Interruptted");
    fflush(NULL);
    exit(0);
}

void alarmclock()
{
    printf("Time up!!!\n");
    exit(0);
}

/*************************************************************************

> File Name: sigcatch.c

> Author: VOID_133

> ###################

> Mail: ###################

> Created Time: Thu 26 May 2016 09:03:32 AM CST

************************************************************************/

#include<stdio.h>

#include<signal.h>

#include<unistd.h>

#include<stdlib.h>

typedef struct sigaction Action;

void interrupt();

void alarmclock();

int main(void)

{

sigset_t set;

sigemptyset(&set);

sigaddset(&set, SIGALRM);

Action sigact;

sigact.sa_mask = set;

sigact.sa_handler = alarmclock;

sigaction(SIGALRM, &sigact, NULL);

sigemptyset(&set);

sigaddset(&set, SIGINT);

sigact.sa_mask = set;

sigact.sa_handler = interrupt;

sigaction(SIGINT, &sigact, NULL);

//This is useless, non-realtime signal will be sent only once

printf("The program will sleep for 3 secs...\n");

while(1)

{

alarm(3);

pause();

}

return 0;

}

void interrupt()

{

printf("0.0 Interruptted");

fflush(NULL);

exit(0);

}

void alarmclock()

{

printf("Time up!!!\n");

exit(0);

}

对于一个信号的处理方式的描述的结构体叫做 sigaction , sigaction也是改变信号相应响应方式的函数, 通过设定 sigaction 的 sa_handler字段, 再调用sigaction函数, 就可以修改相应的信号的处理方式了

可重入将在下一篇文章进行介绍

[go]heka插件开发

五月 25, 2016 VOID001 Comments 4 comments

官方文档传送门

Heka是一个用于进行DataProcessing的一个系统, 具有高度的扩展性, 官方自称为 Swiss Army Knife 型的工具

heka通过一系列插件进行工作heka支持的插件种类有

Input
Splitters
Decoders
Filters
Encoders
Outputs

六种插件 , 对于不同插件的作用, 可以在官方文档内很容易的找到, 这里就不做说明了

开发一个Input插件

heka语言的插件大部分使用 golang 开发, 下面是一个使用heka实现一个input插件, 从文件读入数据的例子

使插件能够被heka识别(注册插件到heka中)

首先, go的开发要保证GOPATH配置正确, heka安装好之后, 为heka开发插件的默认GOPATH是 /path/to/your/heka-src/build/heka/ 如果不将gopath设置为这个, 就会找不到要导入的heka有关的包

为了保证我们对插件开发的可视化, 每一步开发都能看到结果是最好的, 因而我们先尝试开发一个最最简单的插件, 只让这个插件注册到heka中, 能够确定这个插件被加载即可

参考文档:

[1]http://hekad.readthedocs.io/en/v0.10.0/developing/plugin.html#registering-your-plugin
[2]http://hekad.readthedocs.io/en/v0.10.0/installing.html#building-hekad-with-external-plugins

首先我们在新的GOPATH下的src/github.com/下建立自己的工作区和包目录 (我这里以./VOID001/inputplug 为例)

然后我们定义一个type 这个type就是我们的Plugin的type, 这里起名为SampleInput, 其他的名字也都可以 , 这个地方对命名没有什么约束, 下面有对名字有约束的地方, 会进行说明. 然后将这个type实现为Plugin interface(即将 Plugin 这个interface对应的全部方法实现在type上)

type SampleInputPlug struct {
	filePath string
}

func (t *SampleInputPlug) Init(config interface{}) error {
	return nil
}

type SampleInputPlug struct {

filePath string

}

func (t *SampleInputPlug) Init(config interface{}) error {

return nil

}

这样, 就已经实现了一个最基本的Plugin ,SampleInputPlug 指针实现了Plugin这个接口, 然后通过查看 [1] 我们加入这样一句话到Init中, 将我们的Plugin注册到heka中(这里是有两个坑的, 马上就会发现了 = = )

func (t *SampleInputPlug) Init(config interface{}) error {
	pipeline.RegisterPlugin("SampleInputPlug", func() interface{} { return new(SampleInputPlug) })	
	return nil
}

func (t *SampleInputPlug) Init(config interface{}) error {

pipeline.RegisterPlugin("SampleInputPlug", func() interface{} { return new(SampleInputPlug) })

return nil

}

好, 我们go build, 恩没有什么问题, 下一步就是把这个插件给编译到heka中(吐槽: go只支持静态编译, 所以每当新开发一个插件都需要对heka进行一次重新编译=A=很不爽)

根据[2]我们修改 /path/to/your/heka/cmake 下面的 plugin_loader.cmake 添加如下这一行(注:我目前还没有找到添加本地的git repo然后直接clone本地的gitrepo进行编译的方法 , 不过我认为肯定是有的, 目前先不管这个) , 在添加的时候要保证你的这个plugin所在的目录(即inputplug)被添加到了一个 git repo并push到了github上

add_external_plugin(git https://github.com/VOID001/inputplug master)

1	add_external_plugin(git https://github.com/VOID001/inputplug master)

好了,然后在heka根目录执行 source ./build.sh 然后新建一个heka配置文件 sample.toml 里面写上

[SampleInputPlug]

1	[SampleInputPlug]

好了, 然后我们运行 ./hekad -config sample.toml …. 诶 (#°Д°) 为毛出错了!!! (好了即将进入填坑时间)

╭─void001@VOID_PC ~/GitRepos/heka/build/heka/bin  ‹dev› 
╰─➤  ./hekad -config samp.toml
2016/05/25 10:55:11 Pre-loading: [SampleInputPlug]
2016/05/25 10:55:11 No registered plugin type: SampleInputPlug
2016/05/25 10:55:11 Pre-loading: [NullSplitter]
2016/05/25 10:55:11 Loading: [NullSplitter]
2016/05/25 10:55:11 Pre-loading: [ProtobufDecoder]
2016/05/25 10:55:11 Loading: [ProtobufDecoder]
2016/05/25 10:55:11 Pre-loading: [ProtobufEncoder]
2016/05/25 10:55:11 Loading: [ProtobufEncoder]
2016/05/25 10:55:11 Pre-loading: [TokenSplitter]
2016/05/25 10:55:11 Loading: [TokenSplitter]
2016/05/25 10:55:11 Pre-loading: [PatternGroupingSplitter]
2016/05/25 10:55:11 Loading: [PatternGroupingSplitter]
2016/05/25 10:55:11 Pre-loading: [HekaFramingSplitter]
2016/05/25 10:55:11 Loading: [HekaFramingSplitter]
2016/05/25 10:55:11 Error reading config:  Empty configuration, exiting.

╭─void001@VOID_PC ~/GitRepos/heka/build/heka/bin ‹dev›

╰─➤ ./hekad -config samp.toml

2016/05/25 10:55:11 Pre-loading: [SampleInputPlug]

2016/05/25 10:55:11 No registered plugin type: SampleInputPlug

2016/05/25 10:55:11 Pre-loading: [NullSplitter]

2016/05/25 10:55:11 Loading: [NullSplitter]

2016/05/25 10:55:11 Pre-loading: [ProtobufDecoder]

2016/05/25 10:55:11 Loading: [ProtobufDecoder]

2016/05/25 10:55:11 Pre-loading: [ProtobufEncoder]

2016/05/25 10:55:11 Loading: [ProtobufEncoder]

2016/05/25 10:55:11 Pre-loading: [TokenSplitter]

2016/05/25 10:55:11 Loading: [TokenSplitter]

2016/05/25 10:55:11 Pre-loading: [PatternGroupingSplitter]

2016/05/25 10:55:11 Loading: [PatternGroupingSplitter]

2016/05/25 10:55:11 Pre-loading: [HekaFramingSplitter]

2016/05/25 10:55:11 Loading: [HekaFramingSplitter]

2016/05/25 10:55:11 Error reading config: Empty configuration, exiting.

这个错误提示我们这个插件没有被注册 = = 仔细阅读文档之后发现我们的RegisterPlugin的第一个参数字串是不符合要求的

The name value should be a unique identifier for your plugin, and it should end in one of “Input”, “Splitter”, “Decoder”, “Filter”, “Encoder”, or “Output”, depending on the plugin type.

好, 我们把名字改成 SampleInput 去掉后面的Plug ,然后修改我们的toml配置文件 & 重新编译,运行还是不行(#°Д°)

这次是一个很无语的错误 = = 我们应该把RegisterPlugin这个写在 init()函数内而不是 Init 函数内 , init()函数是每一个包在被加载的时候最先调用的函数, 我们应该在这里对Plugin进行注册 , 修改后的代码如下:

package sampinput

import (
	"fmt"
	"github.com/mozilla-services/heka/pipeline"
)

type SampleInputPlug struct {
	filePath string
	config   *SampleInputConfig
}

type SampleInputConfig struct {
}

func init() {
	fmt.Println("Init VOID001's Plugin Done")
	pipeline.RegisterPlugin("SampleInput", func() interface{} { return new(SampleInputPlug) })
}

func (t *SampleInputPlug) Init(config interface{}) error {
	return nil
}

package sampinput

import (

"fmt"

"github.com/mozilla-services/heka/pipeline"

)

type SampleInputPlug struct {

filePath string

config *SampleInputConfig

}

type SampleInputConfig struct {

}

func init() {

fmt.Println("Init VOID001's Plugin Done")

pipeline.RegisterPlugin("SampleInput", func() interface{} { return new(SampleInputPlug) })

}

func (t *SampleInputPlug) Init(config interface{}) error {

return nil

}

再次编译, 运行(WTF)看结果, 恩终于成功了~

╰─➤  ./hekad -config samp.toml
Init VOID001's Plugin Done
2016/05/25 11:14:45 Pre-loading: [SampleInput]
2016/05/25 11:14:45 Pre-loading: [TokenSplitter]
2016/05/25 11:14:45 Loading: [TokenSplitter]
2016/05/25 11:14:45 Pre-loading: [PatternGroupingSplitter]
2016/05/25 11:14:45 Loading: [PatternGroupingSplitter]
2016/05/25 11:14:45 Pre-loading: [HekaFramingSplitter]
2016/05/25 11:14:45 Loading: [HekaFramingSplitter]
2016/05/25 11:14:45 Pre-loading: [NullSplitter]
2016/05/25 11:14:45 Loading: [NullSplitter]
2016/05/25 11:14:45 Pre-loading: [ProtobufDecoder]
2016/05/25 11:14:45 Loading: [ProtobufDecoder]
2016/05/25 11:14:45 Pre-loading: [ProtobufEncoder]
2016/05/25 11:14:45 Loading: [ProtobufEncoder]
2016/05/25 11:14:45 Loading: [SampleInput]

╰─➤ ./hekad -config samp.toml

Init VOID001's Plugin Done

2016/05/25 11:14:45 Pre-loading: [SampleInput]

2016/05/25 11:14:45 Pre-loading: [TokenSplitter]

2016/05/25 11:14:45 Loading: [TokenSplitter]

2016/05/25 11:14:45 Pre-loading: [PatternGroupingSplitter]

2016/05/25 11:14:45 Loading: [PatternGroupingSplitter]

2016/05/25 11:14:45 Pre-loading: [HekaFramingSplitter]

2016/05/25 11:14:45 Loading: [HekaFramingSplitter]

2016/05/25 11:14:45 Pre-loading: [NullSplitter]

2016/05/25 11:14:45 Loading: [NullSplitter]

2016/05/25 11:14:45 Pre-loading: [ProtobufDecoder]

2016/05/25 11:14:45 Loading: [ProtobufDecoder]

2016/05/25 11:14:45 Pre-loading: [ProtobufEncoder]

2016/05/25 11:14:45 Loading: [ProtobufEncoder]

2016/05/25 11:14:45 Loading: [SampleInput]

写好Input插件的框架

上面已经将我们的插件注册到 heka中了, 下来就是开始开发一个Input插件 , 我们先实现从文件中读取内容这一个简单的功能

为了实现一个Input插件, 我们必须实现Input接口 , Input接口的定义如下:

type Input interface {
    Run(ir InputRunner, h PluginHelper) (err error)
    Stop()
}

type Input interface {

Run(ir InputRunner, h PluginHelper) (err error)

Stop()

}

我们为*SampleInputPlug 实现这个接口

func (t *SampleInputPlug) Stop() {
	return
}

func (t *SampleInputPlug) Run(ir pipeline.InputRunner, h pipeline.PluginHelper) (err error) {
	return nil
}

func (t *SampleInputPlug) Stop() {

return

}

func (t *SampleInputPlug) Run(ir pipeline.InputRunner, h pipeline.PluginHelper) (err error) {

return nil

}

对于一个Input Plugin 来说, Run这个函数在插件运行的时候就运行了,并且如果不出什么意外的话(比如突然死机, 或者代码写残了导致它崩溃了) 会一直运行到hekad退出 , 当然就算我们的插件跪了, 如果我们让插件具有了无限复活的buff的话~ (参见 Restarting Plugin) 那么还是可以救回来的 ~

Stop函数也就是释放资源, 清理现场, 比如回收打开的文件, 通知goroutine退出等等~

我们先来看Run函数, 这个函数通常要做这样三件事(也就是一个Input Plugin要做的事啦)

获取data
把获取到的data给一个SplitterRunner(这又是什么呢0.0 别急, 一会儿说)
(可选的)Provide a “pack decorator” function to the SplitterRunner to populate the message object with any input-specific information(暂时没看懂, 先放着)

那么我们就开始第一步的实现啦

首先获取数据, 我们通过打开一个文件来获取输入

if _, err := os.Open(t.filePath); err != nil {
	return fmt.Errorf("Open File %s error: %s", t.filePath, err.Error())
}
file, _ := os.Open(t.filePath)

if _, err := os.Open(t.filePath); err != nil {

return fmt.Errorf("Open File %s error: %s", t.filePath, err.Error())

}

file, _ := os.Open(t.filePath)

这样 file 就是一个 io.Reader 接口 , 下一步就是将数据传递给Splitter Runner

SplitterRunner 就是一个用于读取接收到的数据并且传递到 heka pipeline的接口, 通过 InputRunner 自带的函数 NewSplitterRunner 我们可以实现这个功能, 然后我们使用SplitStream这个方法, 读取并切割这个文件流, 下面是具体实现Run的代码

func (t *SampleInputPlug) Run(ir pipeline.InputRunner, h pipeline.PluginHelper) (err error) {
	//Access Data From Outside world
	if _, err := os.Open(t.filePath); err != nil {
		return fmt.Errorf("Open File %s error: %s", t.filePath, err.Error())
	}
	file, _ := os.Open(t.filePath)
	sr := ir.NewSplitterRunner("")
	for {
		sr.SplitStream(file, nil)
	}
	return nil
}

func (t *SampleInputPlug) Run(ir pipeline.InputRunner, h pipeline.PluginHelper) (err error) {

//Access Data From Outside world

if _, err := os.Open(t.filePath); err != nil {

return fmt.Errorf("Open File %s error: %s", t.filePath, err.Error())

}

file, _ := os.Open(t.filePath)

sr := ir.NewSplitterRunner("")

for {

sr.SplitStream(file, nil)

}

return nil

}

注意:

sr := ir.NewSplitterRunner("")

1	sr := ir.NewSplitterRunner("")

这一句一定要放在循环外, 放在循环里的话, 会导致内存被分配的sr占满(电脑卡死两次的哭着告诉你)

好,然后我们简单修改一下配置文件 sample.toml 让我们能看到我们的修改的效果

[SampleInput]
filepath="/tmp/text"

[PayloadEncoder]
append_newlines = false

[LogOutput]
message_matcher = "TRUE"
encoder = "PayloadEncoder"

[SampleInput]

filepath="/tmp/text"

[PayloadEncoder]

append_newlines = false

[LogOutput]

message_matcher = "TRUE"

encoder = "PayloadEncoder"

然后我们执行 ./hekad -config /tmp/text (text 已经存在) 然后 , 恩, 我们要的输出看到啦 ~~~

好的现在停止heka(Ctrl + C) 结果发现, 没有办法停止(#°Д°)

^C2016/05/25 19:28:16 Shutdown initiated.
2016/05/25 19:28:16 Stop message sent to input 'SampleInput'

1 2	^C2016/05/25 19:28:16 Shutdown initiated. 2016/05/25 19:28:16 Stop message sent to input 'SampleInput'

一直卡在这里….. = = 好吧看来我们还要继续

为我们的Input插件实现退出功能

在插件退出的时候会call Stop函数 ,我们Run实现的是一个forever loop , 一直读取数据直到hekad退出, 为了通知Run退出, 我们需要实现一个 channel , 这里在我们的SampleInputPlug内实现了一个 stopMsg 成员, 类型为 chan interface{} 注意: 在使用你的 channel前一定要make make make!!!!!(又被坑一波的我 = =) 然后我们用 select , 非阻塞的查询信号的到来 , 具体的代码如下~~

import (
	"errors"
	"fmt"
	"github.com/mozilla-services/heka/pipeline"
	"os"
)

type SampleInputPlug struct {
	filePath string
	config   *SampleInputConfig
	stopMsg  chan interface{}
}

type SampleInputConfig struct{}

func init() {
	pipeline.RegisterPlugin("SampleInput", func() interface{} { return new(SampleInputPlug) })
}

func (t *SampleInputPlug) Init(config interface{}) error {
	var ok bool
	conf := config.(pipeline.PluginConfig)
	if t.filePath, ok = conf["filepath"].(string); !ok {
		return errors.New("Now 'filepath' supported")
	}
	fmt.Println("Init Complete")
	t.stopMsg = make(chan interface{})
	return nil
}

func (t *SampleInputPlug) Stop() {
	fmt.Println("Stop Plugin")
	t.stopMsg <- struct{}{}
	return
}

func (t *SampleInputPlug) Run(ir pipeline.InputRunner, h pipeline.PluginHelper) (err error) {
	//Access Data From Outside world
	if _, err := os.Open(t.filePath); err != nil {
		return fmt.Errorf("Open File %s error: %s", t.filePath, err.Error())
	}
	file, _ := os.Open(t.filePath)
	sr := ir.NewSplitterRunner("")
	//Send it to heka pipeline
	for {
		sr.SplitStream(file, nil)
		//Check wheter stop signal is sent
		select {
		case <-t.stopMsg:
			fmt.Println("Runner get the Stop Message")
			return nil
		default:
		}
	}
	return fmt.Errorf("Fatal Error. Plugin Exit Unexpectedly")
}

import (

"errors"

"fmt"

"github.com/mozilla-services/heka/pipeline"

"os"

)

type SampleInputPlug struct {

filePath string

config *SampleInputConfig

stopMsg chan interface{}

}

type SampleInputConfig struct{}

func init() {

pipeline.RegisterPlugin("SampleInput", func() interface{} { return new(SampleInputPlug) })

}

func (t *SampleInputPlug) Init(config interface{}) error {

var ok bool

conf := config.(pipeline.PluginConfig)

if t.filePath, ok = conf["filepath"].(string); !ok {

return errors.New("Now 'filepath' supported")

}

fmt.Println("Init Complete")

t.stopMsg = make(chan interface{})

return nil

}

func (t *SampleInputPlug) Stop() {

fmt.Println("Stop Plugin")

t.stopMsg <- struct{}{}

return

}

func (t *SampleInputPlug) Run(ir pipeline.InputRunner, h pipeline.PluginHelper) (err error) {

//Access Data From Outside world

if _, err := os.Open(t.filePath); err != nil {

return fmt.Errorf("Open File %s error: %s", t.filePath, err.Error())

}

file, _ := os.Open(t.filePath)

sr := ir.NewSplitterRunner("")

//Send it to heka pipeline

for {

sr.SplitStream(file, nil)

//Check wheter stop signal is sent

select {

case <-t.stopMsg:

fmt.Println("Runner get the Stop Message")

return nil

default:

}

return fmt.Errorf("Fatal Error. Plugin Exit Unexpectedly")

}

好了, 这样我们的插件也能正常退出啦 , 这次就到这里 ~~

Draft

最近真的压力山大, 好多事情在一起需要解决,而且重要程度都是同样重要, 没有办法取舍任何一项任务

不过I am VOID

Just Keep VOID

voidisprogramer.com 全站启用HTTPS啦~

五月 16, 2016 VOID001 Comments 4 comments

域名备案总算OK了(生无可恋的眼神) 那么下一件事肯定是给我的WOWO启用HTTPS啦~

首先申请了一个免费证书大家搜索WoSign 免费证书 , 就可以找到能用两年的免费SHA256加密的合法证书哦

然后给我的httpd开启https服务

yum install mod_ssl #Install httpd-ssl mod first

1	yum install mod_ssl #Install httpd-ssl mod first

因为之前没有安装mod_ssl,先安装好, 然后在/etc/httpd.conf/下就有一个默认生成好的配置文件 ssl.conf我们打开, 修改相应的SSL证书路径, 设置为我们从上面获取到的证书的路径,然后重启Apache, 通过https 访问https://voidisprogramer.com 发现OK~….. 不过同时也发现了, 竟然全站CSS还有JS图片都没法加载出来QAQ, Chrome F12看一下, 发现这些资源还都是用https://120.27.97.96这个路径去加载的,难怪加载不出来= =(证书只绑定了https://voidisprogramer.com 和 https://www.voidisprogramer.com两个域名因而= =IP直接访问肯定不行啦) 这个问题倒不难解决, 去Wordpress后台, 的设置, 把WordpressURL, 和站点URL都改成了 https://voidisprogramer.com即可~ [然后我发现就算改完首页banner图还有背景图的URL也没变,于是手动更新一下这两个图片的URL, 后来我发现如果把下面的步骤搞完这个根本不用管的说-0-]

恩~~ https域名访问看来没问题了~~ 那么试试http可以访问么~ , 果然= =!http访问的话,不会定向到https下, 于是乎,下一个要解决的问题就是, 如何把http 还有 IP的直接访问，都定向到https去，这个使用了rewrite mod + NameVirtualHost的限制来实现. 首先, 我们在 /etc/httpd/conf.d/下面新建一个config文件”redirect-direct-ip-visit.conf” 然后加入如下配置:

NameVirtualHost 120.27.97.96
<VirtualHost 120.27.97.96>
        DocumentRoot /var/www/html/redirect-ip-http-to-https/
        ServerName 120.27.97.96
</VirtualHost>

NameVirtualHost 120.27.97.96

DocumentRoot /var/www/html/redirect-ip-http-to-https/

ServerName 120.27.97.96

</VirtualHost>

这里,我们将所有通过IP的访问, 把他们的DocumentRoot设置为了非wordpress站点所在的DocumentRoot,然后在这个目录里新建一个 .htaccess对URL进行重写, 即可完成重定向, 重写的规则如下:

RewriteEngine On
RewriteCond /var/www/html/%{REQUEST_FILENAME} !-f
RewriteRule ^(.+) https://voidisprogramer.com/$1
RewriteRule https://voidisprogramer.com/index.html https://voidisprogramer.com

RewriteEngine On

RewriteCond /var/www/html/%{REQUEST_FILENAME} !-f

RewriteRule ^(.+) https://voidisprogramer.com/$1

RewriteRule https://voidisprogramer.com/index.html https://voidisprogramer.com

这几句的意思一点点解释一下

首先开启了Rewrite功能,

然后如果用户访问的时候,遇到找不到的文件(通过IP,还有http访问必然找不到, 因为在相应的DocumentRoot里只有.htaccess这一个文件,其他的任何请求都是找不到的, ) 这时候, 将请求的URL重定向为 https://voidisprogramer.com/下面的相应文件, 这里最后一句话 ,是因为 ,如果直接访问http://voidisprogramer.com那么, 不加最后一行的配置的话, 会导致重写规则定向到https://voidisprogramer.com/index.html, 这是一个不存在的路由, 因而, 将这个请求特殊处理一下, 如果定向到了index.html那么就直接定向到/

好了~~~ 到了这里, 访问 IP/XXX.html 还是 http:XXX/XXX.html都已经转到 https://XXX/XXX.html了 ~~ 看起来好像没什么问题了呢 QWQ, 不过当我打开Chrome Console,发现竟然还有问题 = =!

QQQ

竟然是萌萌的夏娜酱的gif URL依旧是 Direct IP的QAQ!! 这个夏娜酱是通过一个插件 “伪春菜” 诞生的, 于是我去尝试修改伪春菜的后台Config,想着,给夏娜的URL换一个地址不就可以了么然而- – 让我大吃一鲸的是! 竟然没有提供更换皮肤地址的选项!!!!

ChiJing

当然可以选择删掉这个伪春菜再建一个 = = 然而!! 我怎么可能做出那种丧尽天良的事情! 于是乎, 开始从数据库入手

打开wordpress的数据库,看看有没有weichuncai这个插件的相应数据表吧 ,然后我看到的是这样的:

MariaDB [wordpress]> show tables;
+----------------------------+
| Tables_in_wordpress        |
+----------------------------+
| ahc_browsers               |
| ahc_countries              |
| ahc_hits                   |
| ahc_keywords               |
| ahc_recent_visitors        |
| ahc_refering_sites         |
| ahc_search_engine_crawlers |
| ahc_search_engines         |
| ahc_searching_visits       |
| ahc_title_traffic          |
| ahc_visitors               |
| ahc_visits_time            |
| wp_commentmeta             |
| wp_comments                |
| wp_dxgames                 |
| wp_dxservers               |
| wp_links                   |
| wp_options                 |
| wp_pmxi_files              |
| wp_pmxi_history            |
| wp_pmxi_imports            |
| wp_pmxi_posts              |
| wp_pmxi_templates          |
| wp_postmeta                |
| wp_posts                   |
| wp_rpg_races               |
| wp_statistics_exclusions   |
| wp_statistics_historical   |
| wp_statistics_pages        |
| wp_statistics_search       |
| wp_statistics_useronline   |
| wp_statistics_visit        |
| wp_statistics_visitor      |
| wp_term_relationships      |
| wp_term_taxonomy           |
| wp_termmeta                |
| wp_terms                   |
| wp_usermeta                |
| wp_users                   |
| wsw_404_log                |
+----------------------------+

MariaDB [wordpress]> show tables;

+----------------------------+

| Tables_in_wordpress |

+----------------------------+

| ahc_browsers |

| ahc_countries |

| ahc_hits |

| ahc_keywords |

| ahc_recent_visitors |

| ahc_refering_sites |

| ahc_search_engine_crawlers |

| ahc_search_engines |

| ahc_searching_visits |

| ahc_title_traffic |

| ahc_visitors |

| ahc_visits_time |

| wp_commentmeta |

| wp_comments |

| wp_dxgames |

| wp_dxservers |

| wp_links |

| wp_options |

| wp_pmxi_files |

| wp_pmxi_history |

| wp_pmxi_imports |

| wp_pmxi_posts |

| wp_pmxi_templates |

| wp_postmeta |

| wp_posts |

| wp_rpg_races |

| wp_statistics_exclusions |

| wp_statistics_historical |

| wp_statistics_pages |

| wp_statistics_search |

| wp_statistics_useronline |

| wp_statistics_visit |

| wp_statistics_visitor |

| wp_term_relationships |

| wp_term_taxonomy |

| wp_termmeta |

| wp_terms |

| wp_usermeta |

| wp_users |

| wsw_404_log |

+----------------------------+

= = 泥煤啊! weichuncai这个插件没有单独一个数据表啊擦! (通过这里来看貌似大部分插件都没有单独一个数据表) ,没办法 , 我不可能一个个表去查看, 而且,就算能查找, 该查哪个字段也是不知道的TAT,索性去看weichuncai的源码了

在wordpress目录下找到 wp-content/plugins/weichuncai 这就是这个插件的目录了 ~~ 其中的 sm-options.php,通过查看, 我们发现 ,用户自己定义的URL地址是通过一个叫做 $wcc[‘userdefccs’]这样一个变量保存的, 然后查看了一下, 这个变量是如何被获取到的发现是通过49行

$wcc = get_option('sm-weichuncai');

1	$wcc = get_option('sm-weichuncai');

这里得到的, 然后去找get_option这个函数,发现不是weichuncai插件定义的函数, 于是查找了一下Wordpress的开发者文档中, 找到了对应的说明 https://codex.wordpress.org/Function_Reference/get_option 根据Document所说,找到了wp-includes/option.php. 这个文件进行查看, 通过查看这个文件, 查看到了, get_options是通过查看数据库内的 $wpdb->options表 , 获取相应的options信息返回给用户,

$row = $wpdb->get_row( $wpdb->prepare( "SELECT option_value FROM $wpdb->options WHERE option_name = %s LIMIT 1", $option ) );

1	$row = $wpdb->get_row( $wpdb->prepare( "SELECT option_value FROM $wpdb->options WHERE option_name = %s LIMIT 1", $option ) );

这里出现了wpdb这个对象, 感觉这个对象应该是很重要的一个对象, 因而使用grep查找了这个对象在哪里定义的, 找到了 wp-db.php 这个文件, 打开发现是$wpdb整个类的定义, 然后, 又在options这个定义的附近发现了注释: “WordPress Options Table” , 这个应该就是$wpdb->options对应的表了~~ 也就是伪春菜的配置信息所在的表, 再查看一次数据库内的数据表发现有一个表名为”wp_options” 于是选中这个表, 查询我们要的信息:

SELECT * from wp_options where option_name = 'sm-weichuncai';

1	SELECT * from wp_options where option_name = 'sm-weichuncai';

这里的option_name列是通过desc wp_options得到的表结构,进而得到列名, 然后, 查询的字段是通过, 上面伪春菜插件中的代码 get_options(‘sm-weichuncai’) 得到的, 查询后,果然得到了想要的结果~~

+-----------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| option_id | option_name   | option_value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | autoload |
+-----------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
|       209 | sm-weichuncai | a:11:{s:6:"notice";s:26:"Hmm,, 菠萝包超好吃!!";s:9:"adminname";s:0:"";s:8:"isnotice";N;s:4:"ques";a:5:{i:0;s:9:"早上好";i:1;s:9:"中午好";i:2;s:9:"下午好";i:3;s:9:"晚上好";i:4;s:6:"晚安";}s:3:"ans";a:5:{i:0;s:12:"早上好～";i:1;s:12:"中午好～";i:2;s:12:"下午好～";i:3;s:12:"晚上好～";i:4;s:9:"晚安～";}s:8:"lifetime";a:4:{s:8:"rakutori";i:1459067732;s:4:"neko";i:1459067732;s:11:"chinese_moe";i:1459067732;s:5:"Shana";i:1459068272;}s:3:"ccs";a:3:{i:0;s:8:"rakutori";i:1;s:4:"neko";i:2;s:11:"chinese_moe";}s:10:"defaultccs";s:16:"userdefccs_Shana";s:5:"foods";a:2:{i:0;s:9:"金坷垃";i:1;s:9:"咸梅干";}s:6:"eatsay";a:2:{i:0;s:45:"吃了金坷垃，一刀能秒一万八～！";i:1;s:42:"吃咸梅干，变超人！哦耶～～～";}s:10:"userdefccs";a:1:{s:5:"Shana";a:2:{s:4:"name";s:5:"Shana";s:4:"face";a:3:{i:0;s:79:"http://120.27.97.96/wp-content/uploads/2016/03/u20925666171375876562fm23gp0.gif";i:1;s:0:"";i:2;s:0:"";}}}}                                                                           | yes      |
+-----------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+

+-----------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+

| 209 | sm-weichuncai | a:11:{s:6:"notice";s:26:"Hmm,, 菠萝包超好吃!!";s:9:"adminname";s:0:"";s:8:"isnotice";N;s:4:"ques";a:5:{i:0;s:9:"早上好";i:1;s:9:"中午好";i:2;s:9:"下午好";i:3;s:9:"晚上好";i:4;s:6:"晚安";}s:3:"ans";a:5:{i:0;s:12:"早上好～";i:1;s:12:"中午好～";i:2;s:12:"下午好～";i:3;s:12:"晚上好～";i:4;s:9:"晚安～";}s:8:"lifetime";a:4:{s:8:"rakutori";i:1459067732;s:4:"neko";i:1459067732;s:11:"chinese_moe";i:1459067732;s:5:"Shana";i:1459068272;}s:3:"ccs";a:3:{i:0;s:8:"rakutori";i:1;s:4:"neko";i:2;s:11:"chinese_moe";}s:10:"defaultccs";s:16:"userdefccs_Shana";s:5:"foods";a:2:{i:0;s:9:"金坷垃";i:1;s:9:"咸梅干";}s:6:"eatsay";a:2:{i:0;s:45:"吃了金坷垃，一刀能秒一万八～！";i:1;s:42:"吃咸梅干，变超人！哦耶～～～";}s:10:"userdefccs";a:1:{s:5:"Shana";a:2:{s:4:"name";s:5:"Shana";s:4:"face";a:3:{i:0;s:79:"http://120.27.97.96/wp-content/uploads/2016/03/u20925666171375876562fm23gp0.gif";i:1;s:0:"";i:2;s:0:"";}}}} | yes |

(太长了就不要在排版意细节了)

然后只要更新option_value数据项~~ 我们就可以更换夏娜酱的URL啦~~~

然而= =!!! 遇到了一个大坑!!! 当使用下面的内容替换上面的option_value后

a:11:{s:6:"notice";s:26:"Hmm,, 菠萝包超好吃!!";s:9:"adminname";s:0:"";s:8:"isnotice";N;s:4:"ques";a:5:{i:0;s:9:"早上好";i:1;s:9:"中午好";i:2;s:9:"下午好";i:3;s:9:"晚上好";i:4;s:6:"晚安";}s:3:"ans";a:5:{i:0;s:12:"早上好～";i:1;s:12:"中午好～";i:2;s:12:"下午好～";i:3;s:12:"晚上好～";i:4;s:9:"晚安～";}s:8:"lifetime";a:4:{s:8:"rakutori";i:1459067732;s:4:"neko";i:1459067732;s:11:"chinese_moe";i:1459067732;s:5:"Shana";i:1459068272;}s:3:"ccs";a:3:{i:0;s:8:"rakutori";i:1;s:4:"neko";i:2;s:11:"chinese_moe";}s:10:"defaultccs";s:16:"userdefccs_Shana";s:5:"foods";a:2:{i:0;s:9:"金坷垃";i:1;s:9:"咸梅干";}s:6:"eatsay";a:2:{i:0;s:45:"吃了金坷垃，一刀能秒一万八～！";i:1;s:42:"吃咸梅干，变超人！哦耶～～～";}s:10:"userdefccs";a:1:{s:5:"Shana";a:2:{s:4:"name";s:5:"Shana";s:4:"face";a:3:{i:0;s:79:"http://occqvoj4t.bkt.clouddn.com/wp-content/uploads/2016/03/u20925666171375876562fm23gp0.gif";i:1;s:0:"";i:2;s:0:"";}}}}

a:11:{s:6:"notice";s:26:"Hmm,, 菠萝包超好吃!!";s:9:"adminname";s:0:"";s:8:"isnotice";N;s:4:"ques";a:5:{i:0;s:9:"早上好";i:1;s:9:"中午好";i:2;s:9:"下午好";i:3;s:9:"晚上好";i:4;s:6:"晚安";}s:3:"ans";a:5:{i:0;s:12:"早上好～";i:1;s:12:"中午好～";i:2;s:12:"下午好～";i:3;s:12:"晚上好～";i:4;s:9:"晚安～";}s:8:"lifetime";a:4:{s:8:"rakutori";i:1459067732;s:4:"neko";i:1459067732;s:11:"chinese_moe";i:1459067732;s:5:"Shana";i:1459068272;}s:3:"ccs";a:3:{i:0;s:8:"rakutori";i:1;s:4:"neko";i:2;s:11:"chinese_moe";}s:10:"defaultccs";s:16:"userdefccs_Shana";s:5:"foods";a:2:{i:0;s:9:"金坷垃";i:1;s:9:"咸梅干";}s:6:"eatsay";a:2:{i:0;s:45:"吃了金坷垃，一刀能秒一万八～！";i:1;s:42:"吃咸梅干，变超人！哦耶～～～";}s:10:"userdefccs";a:1:{s:5:"Shana";a:2:{s:4:"name";s:5:"Shana";s:4:"face";a:3:{i:0;s:79:"http://occqvoj4t.bkt.clouddn.com/wp-content/uploads/2016/03/u20925666171375876562fm23gp0.gif";i:1;s:0:"";i:2;s:0:"";}}}}

结果! 夏娜消失了!!

再去查看数据库,发现刚刚的修改也失效了, 修改没有生效, 设置变成了最初安装时候的设置 = = 看来是因为设置不对,导致自动初始化设置了

经过多次查看原因发现, 在修改字符串的前面还有一个属性”s” 这个属性的值原来是字符串的长度, 在修改了URL之后,也要一起修改这个, 才可以 QAQ 于是将更新的语句换成这个:

UPDATE `wp_options` SET `option_value` = 'a:11:{s:6:"notice";s:26:"Hmm,, 菠萝包超好吃!!";s:9:"adminname";s:0:"";s:8:"isnotice";N;s:4:"ques";a:5:{i:0;s:9:"早上好";i:1;s:9:"中午好";i:2;s:9:"下午好";i:3;s:9:"晚上好";i:4;s:6:"晚安";}s:3:"ans";a:5:{i:0;s:12:"早上好～";i:1;s:12:"中午好～";i:2;s:12:"下午好～";i:3;s:12:"晚上好～";i:4;s:9:"晚安～";}s:8:"lifetime";a:4:{s:8:"rakutori";i:1459067732;s:4:"neko";i:1459067732;s:11:"chinese_moe";i:1459067732;s:5:"Shana";i:1459068272;}s:3:"ccs";a:3:{i:0;s:8:"rakutori";i:1;s:4:"neko";i:2;s:11:"chinese_moe";}s:10:"defaultccs";s:16:"userdefccs_Shana";s:5:"foods";a:2:{i:0;s:9:"金坷垃";i:1;s:9:"咸梅干";}s:6:"eatsay";a:2:{i:0;s:45:"吃了金坷垃，一刀能秒一万八～！";i:1;s:42:"吃咸梅干，变超人！哦耶～～～";}s:10:"userdefccs";a:1:{s:5:"Shana";a:2:{s:4:"name";s:5:"Shana";s:4:"face";a:3:{i:0;s:87:"http://occqvoj4t.bkt.clouddn.com/wp-content/uploads/2016/03/u20925666171375876562fm23gp0.gif";i:1;s:0:"";i:2;s:0:"";}}}}' WHERE `wp_options`.`option_id` = 209;

UPDATE `wp_options` SET `option_value` = 'a:11:{s:6:"notice";s:26:"Hmm,, 菠萝包超好吃!!";s:9:"adminname";s:0:"";s:8:"isnotice";N;s:4:"ques";a:5:{i:0;s:9:"早上好";i:1;s:9:"中午好";i:2;s:9:"下午好";i:3;s:9:"晚上好";i:4;s:6:"晚安";}s:3:"ans";a:5:{i:0;s:12:"早上好～";i:1;s:12:"中午好～";i:2;s:12:"下午好～";i:3;s:12:"晚上好～";i:4;s:9:"晚安～";}s:8:"lifetime";a:4:{s:8:"rakutori";i:1459067732;s:4:"neko";i:1459067732;s:11:"chinese_moe";i:1459067732;s:5:"Shana";i:1459068272;}s:3:"ccs";a:3:{i:0;s:8:"rakutori";i:1;s:4:"neko";i:2;s:11:"chinese_moe";}s:10:"defaultccs";s:16:"userdefccs_Shana";s:5:"foods";a:2:{i:0;s:9:"金坷垃";i:1;s:9:"咸梅干";}s:6:"eatsay";a:2:{i:0;s:45:"吃了金坷垃，一刀能秒一万八～！";i:1;s:42:"吃咸梅干，变超人！哦耶～～～";}s:10:"userdefccs";a:1:{s:5:"Shana";a:2:{s:4:"name";s:5:"Shana";s:4:"face";a:3:{i:0;s:87:"http://occqvoj4t.bkt.clouddn.com/wp-content/uploads/2016/03/u20925666171375876562fm23gp0.gif";i:1;s:0:"";i:2;s:0:"";}}}}' WHERE `wp_options`.`option_id` = 209;

这次终于对了!!!!夏娜酱回来了 ~~ 之前的设置也都保留了, 撒花QAQ(到现在还没吃饭一直在搞这个这才叫真爱吧!!!

[C Linux 内核] 文件系统 #2

五月 13, 2016 VOID001 Comments 1 comment

上文我们已经通过一个ext2文件系统的例子详细的查看了文件系统的各种属性, 数据是如何存储的

不过很显然有一个疑问: 如果一个inode只能存15个Block的索引项, 即之恩那个指向15个Datablock每一个Datablock大小又只是1KB, 那么一个文件的maxsize不就只有15KB了么, 下面就通过这个事实 , 对Linux ext文件系统的数据块寻址进行介绍

数据块寻址

对于我们上面那个例子里的ext2文件系统来说, 只有前面 12个块是直接指向一个存放数据的Datablock, 而 Block 12, 13, 14则都是间接寻址块

所谓间接寻址块, 就是将本来用作存储文件数据的Datablock用作inode索引(即在这个Datablock中存储的是一个个Block index记录)

在上述sample filesystem中 , 总共有三个间接寻址块, Block 12 13 14 他们之间的关系如下图所示

其中浅蓝色表示真正存储数据的Datablock, 而浅绿色, 深绿色, 橙色的则表示一级, 二级, 三级Block索引, 以一级索引为例, 以及Block索引指向的是一个 Datablock, 这个Datablock 存放的是Block Index,而二级Block索引则指向一个Datablock ,这个Datablock存的是一系列的一级索引, 然后每一个一级索引又指向一个存放着Block Index的Datablock,以此类推 ,通过这个方法就大大扩充了一个文件的容量

文件&目录的操作函数

以下这些函数都是通过读取inode block, datablock 里相应的信息, 返回给调用者 / 对文件 or 目录进行操作

只对inode进行操作的(取出/修改)
- lstat, stat, fstat(取出)
- access (取出)
- chmod, fchmod(修改 st_mode)
- chown, fchown, lchown(修改user, group)
- utime(修改 atime, mtime)
同时对inode, datablock进行操作的
- truncate, ftruncate(截断文件(既可以向短截断,也可以向长截断))
- symlink(符号链接, 创建一个新的inode, st_mode字段文件类型为symlink, 原文件路径根据长度可能保存在inode内或者分配一个datablock来存
- unlink
- rename
- rmdir
对目录进行操作的专用函数
- opendir
- readdir
- closedir

关于每一个命令具体对应的磁盘上的inode, datablock的操作省略, 在《Linux C编程一站式学习》Chap29 中有介绍，本文也是对其的一个整理，总结

下面给出一个模拟 ls -R 的实现代码参考K&R C

/*************************************************************************
    > File Name: dir_walk_example.c
    > Author: VOID_133
    > A small program demos the process "ls -R", potential buggy
    > Mail: ################### 
    > Created Time: Mon 09 May 2016 09:24:56 PM CST
 ************************************************************************/
#include<stdio.h>
#include<string.h>
#include<stdlib.h>
#include<unistd.h>
#include<sys/file.h>
#include<sys/types.h>
#include<sys/stat.h>
#include<dirent.h>
#include<fcntl.h>
#include<errno.h>


#define MAX_PATH 2048
struct stat fs_stat;
VFS
void usage();
void get_size(char *filename);

int main(int argc, char *argv[])
{
    if(argc < 2)
    {
        usage();
    }
    else
    {
        while(--argc > 0)
            get_size(*++argv);
    }
}

void dir_walk(char *dir, void (*fsz)(char *))
{
    char file_name[MAX_PATH];
    DIR *dfd;
    struct dirent* pentry;
    printf("entering %s...\n", dir);
    if((dfd = opendir(dir)) == NULL)
    {
        perror("opendir");
        //exit(errno);
    }
    else
    {
        while((pentry = readdir(dfd)) != NULL)
        {
            if(!strcmp(pentry->d_name, ".") || !strcmp(pentry->d_name, ".."))
            {
                printf("%8ld %s\n", 0, pentry->d_name);
                continue;
            }
            if(strlen(dir) + strlen(pentry->d_name) + 2 > MAX_PATH)
            {
                fprintf(stderr, "dir_walk name %s/%s is too long", dir, pentry->d_name);
            }
            if(!strcmp(dir,"/"))
                sprintf(file_name, "/%s", pentry->d_name);
            else
                sprintf(file_name, "%s/%s", dir, pentry->d_name);
            fsz(file_name);
        }
        closedir(dfd);
    }
    return ;
}

void get_size(char *file_name)
{
    int retval = 0xdeadbeef;
    retval = lstat(file_name, &fs_stat);
    if(retval < 0)
        perror("stat");
    else
    {
        mode_t mode = fs_stat.st_mode;
        if(((mode & S_IFMT) == S_IFDIR) && ((mode & S_IFMT) != S_IFLNK))
        {
            dir_walk(file_name, get_size);
        }
        else
        {
            printf("%8ld %s\n", fs_stat.st_size, file_name);
        }
    }
}

void usage()
{
    printf("Usage: myls directory_name\n");
}

/*************************************************************************

> File Name: dir_walk_example.c

> Author: VOID_133

> A small program demos the process "ls -R", potential buggy

> Mail: ###################

> Created Time: Mon 09 May 2016 09:24:56 PM CST

************************************************************************/

#include<stdio.h>

#include<string.h>

#include<stdlib.h>

#include<unistd.h>

#include<sys/file.h>

#include<sys/types.h>

#include<sys/stat.h>

#include<dirent.h>

#include<fcntl.h>

#include<errno.h>

#define MAX_PATH 2048

struct stat fs_stat;

VFS

void usage();

void get_size(char *filename);

int main(int argc, char *argv[])

{

if(argc < 2)

{

usage();

}

else

{

while(--argc > 0)

get_size(*++argv);

}

void dir_walk(char *dir, void (*fsz)(char *))

{

char file_name[MAX_PATH];

DIR *dfd;

struct dirent* pentry;

printf("entering %s...\n", dir);

if((dfd = opendir(dir)) == NULL)

{

perror("opendir");

//exit(errno);

}

else

{

while((pentry = readdir(dfd)) != NULL)

{

if(!strcmp(pentry->d_name, ".") || !strcmp(pentry->d_name, ".."))

{

printf("%8ld %s\n", 0, pentry->d_name);

continue;

}

if(strlen(dir) + strlen(pentry->d_name) + 2 > MAX_PATH)

{

fprintf(stderr, "dir_walk name %s/%s is too long", dir, pentry->d_name);

}

if(!strcmp(dir,"/"))

sprintf(file_name, "/%s", pentry->d_name);

else

sprintf(file_name, "%s/%s", dir, pentry->d_name);

fsz(file_name);

}

closedir(dfd);

}

return ;

}

void get_size(char *file_name)

{

int retval = 0xdeadbeef;

retval = lstat(file_name, &fs_stat);

if(retval < 0)

perror("stat");

else

{

mode_t mode = fs_stat.st_mode;

if(((mode & S_IFMT) == S_IFDIR) && ((mode & S_IFMT) != S_IFLNK))

{

dir_walk(file_name, get_size);

}

else

{

printf("%8ld %s\n", fs_stat.st_size, file_name);

}

void usage()

{

printf("Usage: myls directory_name\n");

}

书上说这个代码有bug，可能导致递归死循环，我在这里将书上的一个地方进行了修改，不知道是否解决了书上所说的问题，代码中

retval = stat(file_name, &fs_stat);

1	retval = stat(file_name, &fs_stat);

我改成了

retval = lstat(file_name, &fs_stat);

1	retval = lstat(file_name, &fs_stat);

这样遇到symlink就不会递归进去，不过就算递归进入symlink，如果有一个symlink链接到它的父目录的话，也会由于 too many symlink这样的错误导致程序的停止，不会死循环，所以目前为止还不清楚具体是哪种情况会让上面的代码死循环

VFS

下面说一下 VFS ， Linux支持各种各样的文件系统格式，如 ext2, ext3, ext4, ntfs 等，那么按照上面对文件系统的分析来看，不同文件系统都需要实现一套不同的stat, link, opendir … 等操作相应文件系统的函数，而当将某个文件系统挂载到相应的文件夹下，进去查看之后发现，不同的文件系统在linux 下的表现都是一样的，这就是VFS的效果， Linux内核在各种不同的filesystem格式上做了一个抽象层，使得inode, 目录，等概念成为抽象层的概念

Linux的VFS维护了以下几个数据结构，达到抽象文件系统的效果

super_block
inode
dentry
file

对于每一个打开的文件(FILE结构体)都指向一个dentry结构体，每一个dentry又指向一个inode， inode指向super_block，对于不同的文件系统，他们inode结构体内的 inode_operation 还有 file内的file_operation可能指向不同的操作，而这些对于使用的用户（即我们开发者）在使用系统调用的时候，是透明的，我们只要使用某个操作函数(如stat)然后内核自动会去调用正确的函数将正确的信息返回给用户

文件的引用数,dup&dup2

我们用 open(2)打开的同一个文件享有不同的两个file结构体, 他们保存有不同的文件打开标志(FLAG)和读写位置, 而dup dup2这两个函数, 则提供了可以让两个不同的文件描述符指向同一个file结构体这样的功能 , 如果某一个file结构体被多于一个fd指向, 那么, 这时候, file结构体的引用数就是一个关键的属性了, 引用数表明, 该file被多少个fd指向(引用),当用close关闭这个文件的一个fd的时候, 只要file的引用数不为0, 这个file结构体不会被释放. dup是直接复制一个新的文件描述符出来, 不能指定新fd的值, 而dup2可以指定新的fd的值,如果该fd被打开,则先关闭该新fd,再重新dup它

密码保护：accelercode 回忆录 By NEU20133823(貌似你没有昵称就用这个了233)

五月 12, 2016 VOID001 Comments 要查看留言请输入您的密码。

[C Linux 内核] 文件系统(以ext2为例)

五月 8, 2016 VOID001 Comments 1 comment

ext2文件系统的总体结构

每一个文件系统, 由若干个块组构成, 文件系统的最前面是大小固定(大小不能由用户指定,PC标准规定启动块大小为1KB)的一个启动快(Boot Block) 然后是若干块组, 每个块组内含有的信息为:

超级块(Super Block)
块组描述符表(Group Descriptor Table GDT)
块位图(Block Bitmap)
inode位图(inode Bitmap)
inode表(inode Table)
数据块(Data Block)

其中, 超级块是描述该文件系统(注意,不是该块)的基本信息,如块大小, 文件系统版本号等信息, 可以通过 dumpe2fs

的头部信息查看到superblock信息, 在该文件系统中,每一个块组的头部, 都有一份Superblock信息的copy

块组描述符表, 存储该块组的基本信息,每一个Block group一个

块位图, 每一个bit代表该块组中的一个块, 如果这个块被使用了, 则该bit为1 否则为0

inode位图, 和楼上类似,只不过这次每一个bit代表的是该inode块中的一个块, inode位图单独占一个块

inode表: 存储文件的基本描述信息, 如文件类型, 权限, 大小, 文件的时间戳信息等, 都存储在inode块内, 多个inode块构成inode表, 而且特殊的, 一些不占用磁盘空间的文件 ,如 /dev/tty 这种设备文件, 内存映象文件等, 都只占用inode block,而不占用Data block

数据块: 数据块内数据的存储根据不同filetype有不同的情况:

对于regular file, 文件的数据存储在数据块中
对于目录文件, 该目录下的所有文件名和目录名存在数据块中, 注意, 一个 regular file的文件名不保存在他的数据块中, 而是保存在他目录的数据块中
对于符号链接, 如果目标路径短,则存在inode中, 否则, 要分配一个数据块来保存这个符号链接名
设备文件, FIFO socket等特殊文件没有数据块,只有inode块, 可以通过 ls -l /dev/tty 以及对比 ls -l /etc/shadow 发现二者信息的不同

#ls -lh /dev/tty
crw-rw-rw- 1 root tty 5, 0 May  8 16:01 /dev/tty

#ls -lh /etc/shadow
-rw------- 1 root root 1.6K Apr 26 09:15 /etc/shadow

#ls -lh /dev/tty

crw-rw-rw- 1 root tty 5, 0 May 8 16:01 /dev/tty

#ls -lh /etc/shadow

-rw------- 1 root root 1.6K Apr 26 09:15 /etc/shadow

下面我们就通过一个例子, 来演示一下, ext2文件系统的具体构成

首先, 在你的Linux下新建一个空的 1M大小的文件, 并把他做成一个 ext2 filesystem 具体做法参考下面的代码, 都是基本的命令, 不做解释

#create an empty file
dd if=/dev/zero of=sample_fs bs=4K count=256
#format it to ext2
mkfs.ext2 sample_fs

#create an empty file

dd if=/dev/zero of=sample_fs bs=4K count=256

#format it to ext2

mkfs.ext2 sample_fs

这样之后, 你便得到了一个崭新的ext2文件系统, 它可以像普通的磁盘设备一样被挂载, 下面挂载一下这个文件系统, 看看里面有什么东西

mkdir mountpoint
sudo mount sample_fs mountpoint

1 2	mkdir mountpoint sudo mount sample_fs mountpoint

ls -l 看看信息, 然后umount 这个磁盘我这里给出我改动过后(touch了一个biu 文件之后)的ls -al信息

total 17
drwxr-xr-x 3 root    root     1024 May  7 22:14 .
drwxr-xr-x 3 void001 void001  4096 May  8 17:06 ..
-rw-r--r-- 1 root    root        0 May  7 22:14 biu
drwx------ 2 root    root    12288 May  7 16:02 lost+found

total 17

drwxr-xr-x 3 root root 1024 May 7 22:14 .

drwxr-xr-x 3 void001 void001 4096 May 8 17:06 ..

-rw-r--r-- 1 root root 0 May 7 22:14 biu

drwx------ 2 root root 12288 May 7 16:02 lost+found

你们看到的应该没有 biu 这个文件, 现在我们umount掉这个磁盘, 看看这个磁盘的信息在sample_fs这一个二进制文件中是如何存储的

我们使用 od -tx1 -Ax sample_fs 来dump出整个ext2 filesystem的信息, 然后对照着 dumpe2fs sample_fs 以及 debugfs sample_fs 的信息,来一个byte一个byte的对应查看

首先前面的1024个Byte是Boot block, 因为这不是一个可以启动的文件系统, 所以数据没有意义, 然后是superblock ,然后是 GDT 我们按照这个顺序给出相应的分析在下面的分析文件中, 每一个数据域用” | |” 分割, 对它的解释用逗号”,”分割

000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  # Frist Position is Boot Block
*
000400 |80 00 00 00|00 04 00 00|33 00 00 00|da 03 00 00| # Start of Super Block
inode count = 0x80(128), block count = 0x0400(1024) reserved block count = 0x33(51) free blocks = 0x03da(986)
000410 |75 00 00 00|01 00 00 00|00 00 00 00 00 00 00 00|
free inodes = 0x75(117), first block = 0x01(1), for log use(unused)
000420 |00 20 00 00|00 20 00 00|80 00 00 00|ea a3 2d 57
blocks = 0x2000(8192), fragments = 0x2000(8192), inodes/group = 0x80(128), lastmounttime = 0x572da3ea
000430 |45 a4 2d 57|02 00|ff ff|53 ef|01 00|01 00|00 00|
last write time = 0x572da445, mount count = 0x03(2), max count = 0xffff, magic = 0xEF53, fs_state = 0x01(clean), err behavior = continue, minor rev = 0
000440 |1c a1 2d 57|00 00 00 00|00 00 00 00|01 00 00 00|
last checked 0x572da11c, check interval = 0x00, os_type = 0x00(Linux), major rev = 0x01
000450 |00 00|00 00|0b 00 00 00|80 00|00 00|38 00 00 00|
reserved blocks uid = 0x00, reserved blocks gid = 0x00, first nonreserved inode = 11
(So there are 128 -11 = 117 free inodes at this time (clean and new fs)), inode size = 0x80, block group = 0, compatible features= (0x38)
000460 |02 00 00 00 03 00 00 00|54 75 23 b2 e0 e2 41 3f
compatible features, UUID = （这个不用管little endian)  547523b2-e0e2-413f-9982-579d7a986dc9
000470 99 82 57 9d 7a 98 6d c9| 00 00 00 00 00 00 00 00
UUID, continued, volumn name = NULL
000480 00 00 00 00 00 00 00 00|00 00 00 00|00|00|00 00
last mounted on = NULL, alogrithm usage bitmap, pre_alloc , prealloc, padding 
*
0004c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 00
0004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0004e0 00 00 00 00 00 00 00 00 00 00 00 00 61 eb 9a 73
0004f0 78 b3 41 27 98 04 5a 9f 4b b0 75 2b 01 00 00 00
000500 0c 00 00 00 00 00 00 00 1c a1 2d 57 00 00 00 00
000510 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
000560 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000570 00 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00
000580 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
000800 |06 00 00 00|07 00 00 00|08 00 00 00|da 03|75 00| # Group Description Table(For this sample_fs only one block to store)
block bitmap at 0x6, inode bitmap at 0x7, inode table at 0x8, freeblocks = 0x3da(986), free inodes = 0x75(117)
000810 |02 00|04 00 00 00 00 00 00 00 00 00 00 00 00 00|
directories = 0x2, padding
000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
001800 ff ff ff ff 1f 00 00 00 00 00 00 00 00 00 00 00  # Block Bitmap
001810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
001870 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80
001880 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
*
001c00 ff 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00  # This line is Inode Bitmap (128 Bit already)
ff 07 = 1111 1111 0000 0111 , That is 11 inodes (10 for fs reserve, one(inode no = 11) for lost+found directory, one
(inode no = 2) for root directory)
001c10 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
*
002000 00 00 00 00 00 00 00 00 1c a1 2d 57 1c a1 2d 57
002010 1c a1 2d 57 00 00 00 00 00 00 00 00 00 00 00 00
002020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
002080 |ed 41|00 00|00 04 00 00|40 a1 2d 57|1c a1 2d 57|  # This is the inode block for root dir "/" (Inode 2)
st_mode = 0x41ed = 040755(st_mode contains filetype & permission), user = 0, size = 1024, atime, ctime
002090 |1c a1 2d 57|00 00 00 00|00 00|03 00|02 00 00 00|
mtime, dtime, group = 0000, links = 3("/", and ".", and".."), block_count = 2(512(Sector size = 512) * 2)
0020a0 |00 00 00 00|00 00 00 00|18 00 00 00|00 00 00 00|
flags = 0x00, OS_Information, blocks[0] = 0x18(24)[This means the Block No.24 Contains data for this inode, and we can get the address for Block No.24(0x400 * 24 = 0x6000)], blocks[1] = 0
0020b0 |00 00 00 00|00 00 00 00|00 00 00 00|00 00 00 00|
blocks[2] = 0, [3], [4], [5],...

*
002300 80 81 00 00 00 30 04 04 1c a1 2d 57 1c a1 2d 57
002310 1c a1 2d 57 00 00 00 00 00 00 01 00 08 00 00 00
002320 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
002350 00 00 00 00 00 00 00 00 00 00 00 00 25 00 00 00
002360 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
002500 |c0 41|00 00|00 30 00 00|1c a1 2d 57|1c a1 2d 57|
002510 |1c a1 2d 57|00 00 00 00|00 00|02 00|18 00 00 00|
002520 |00 00 00 00|00 00 00 00|19 00 00 00|1a 00 00 00|
002530 |1b 00 00 00|1c 00 00 00|1d 00 00 00|1e 00 00 00|
002540 |1f 00 00 00|20 00 00 00|21 00 00 00|22 00 00 00|
002550 |23 00 00 00|24 00 00 00|00 00 00 00 00 00 00 00
002560 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006000 [|02 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00 # This is the data block for rootfs "/"( inode no 2, block no 24, type+permission 040755)
[inode = 2, record len = 12(Len(inBytes) of one record, hereis 02 00 00 00 0c 00 01 02 2e 00 00 00), namelen = 1, filetype = 2(directory), content of the file (".")], [Next has the same format as prev record, only present the data:2,12,2,2(dir),".."]
006010 |0c 00|02|02|2e 2e 00 00|][|0b 00 00 00|e8 03|0a|02
006020 6c 6f 73 74 2b 66 6f 75 6e 64 00 00 00 00 00 00
006030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006400 [0b 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00| 
006410 f4 03|02|02|2e 2e 00 00 00 00 00 00 00 00 00 00
006420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00  
006810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
006c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
009000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
009010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
009400 00 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
009410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
100000

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # Frist Position is Boot Block

000400 |80 00 00 00|00 04 00 00|33 00 00 00|da 03 00 00| # Start of Super Block

inode count = 0x80(128), block count = 0x0400(1024) reserved block count = 0x33(51) free blocks = 0x03da(986)

000410 |75 00 00 00|01 00 00 00|00 00 00 00 00 00 00 00|

free inodes = 0x75(117), first block = 0x01(1), for log use(unused)

000420 |00 20 00 00|00 20 00 00|80 00 00 00|ea a3 2d 57

blocks = 0x2000(8192), fragments = 0x2000(8192), inodes/group = 0x80(128), lastmounttime = 0x572da3ea

000430 |45 a4 2d 57|02 00|ff ff|53 ef|01 00|01 00|00 00|

last write time = 0x572da445, mount count = 0x03(2), max count = 0xffff, magic = 0xEF53, fs_state = 0x01(clean), err behavior = continue, minor rev = 0

000440 |1c a1 2d 57|00 00 00 00|00 00 00 00|01 00 00 00|

last checked 0x572da11c, check interval = 0x00, os_type = 0x00(Linux), major rev = 0x01

000450 |00 00|00 00|0b 00 00 00|80 00|00 00|38 00 00 00|

reserved blocks uid = 0x00, reserved blocks gid = 0x00, first nonreserved inode = 11

(So there are 128 -11 = 117 free inodes at this time (clean and new fs)), inode size = 0x80, block group = 0, compatible features= (0x38)

000460 |02 00 00 00 03 00 00 00|54 75 23 b2 e0 e2 41 3f

compatible features, UUID = （这个不用管little endian) 547523b2-e0e2-413f-9982-579d7a986dc9

000470 99 82 57 9d 7a 98 6d c9| 00 00 00 00 00 00 00 00

UUID, continued, volumn name = NULL

000480 00 00 00 00 00 00 00 00|00 00 00 00|00|00|00 00

last mounted on = NULL, alogrithm usage bitmap, pre_alloc , prealloc, padding

0004c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 00

0004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0004e0 00 00 00 00 00 00 00 00 00 00 00 00 61 eb 9a 73

0004f0 78 b3 41 27 98 04 5a 9f 4b b0 75 2b 01 00 00 00

000500 0c 00 00 00 00 00 00 00 1c a1 2d 57 00 00 00 00

000510 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

000560 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

000570 00 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00

000580 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

000800 |06 00 00 00|07 00 00 00|08 00 00 00|da 03|75 00| # Group Description Table(For this sample_fs only one block to store)

block bitmap at 0x6, inode bitmap at 0x7, inode table at 0x8, freeblocks = 0x3da(986), free inodes = 0x75(117)

000810 |02 00|04 00 00 00 00 00 00 00 00 00 00 00 00 00|

directories = 0x2, padding

000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

001800 ff ff ff ff 1f 00 00 00 00 00 00 00 00 00 00 00 # Block Bitmap

001810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

001870 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80

001880 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

001c00 ff 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # This line is Inode Bitmap (128 Bit already)

ff 07 = 1111 1111 0000 0111 , That is 11 inodes (10 for fs reserve, one(inode no = 11) for lost+found directory, one

(inode no = 2) for root directory)

001c10 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

002000 00 00 00 00 00 00 00 00 1c a1 2d 57 1c a1 2d 57

002010 1c a1 2d 57 00 00 00 00 00 00 00 00 00 00 00 00

002020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

002080 |ed 41|00 00|00 04 00 00|40 a1 2d 57|1c a1 2d 57| # This is the inode block for root dir "/" (Inode 2)

st_mode = 0x41ed = 040755(st_mode contains filetype & permission), user = 0, size = 1024, atime, ctime

002090 |1c a1 2d 57|00 00 00 00|00 00|03 00|02 00 00 00|

mtime, dtime, group = 0000, links = 3("/", and ".", and".."), block_count = 2(512(Sector size = 512) * 2)

0020a0 |00 00 00 00|00 00 00 00|18 00 00 00|00 00 00 00|

flags = 0x00, OS_Information, blocks[0] = 0x18(24)[This means the Block No.24 Contains data for this inode, and we can get the address for Block No.24(0x400 * 24 = 0x6000)], blocks[1] = 0

0020b0 |00 00 00 00|00 00 00 00|00 00 00 00|00 00 00 00|

blocks[2] = 0, [3], [4], [5],...

002300 80 81 00 00 00 30 04 04 1c a1 2d 57 1c a1 2d 57

002310 1c a1 2d 57 00 00 00 00 00 00 01 00 08 00 00 00

002320 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

002350 00 00 00 00 00 00 00 00 00 00 00 00 25 00 00 00

002360 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

002500 |c0 41|00 00|00 30 00 00|1c a1 2d 57|1c a1 2d 57|

002510 |1c a1 2d 57|00 00 00 00|00 00|02 00|18 00 00 00|

002520 |00 00 00 00|00 00 00 00|19 00 00 00|1a 00 00 00|

002530 |1b 00 00 00|1c 00 00 00|1d 00 00 00|1e 00 00 00|

002540 |1f 00 00 00|20 00 00 00|21 00 00 00|22 00 00 00|

002550 |23 00 00 00|24 00 00 00|00 00 00 00 00 00 00 00

002560 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006000 [|02 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00 # This is the data block for rootfs "/"( inode no 2, block no 24, type+permission 040755)

[inode = 2, record len = 12(Len(inBytes) of one record, hereis 02 00 00 00 0c 00 01 02 2e 00 00 00), namelen = 1, filetype = 2(directory), content of the file (".")], [Next has the same format as prev record, only present the data:2,12,2,2(dir),".."]

006010 |0c 00|02|02|2e 2e 00 00|][|0b 00 00 00|e8 03|0a|02

006020 6c 6f 73 74 2b 66 6f 75 6e 64 00 00 00 00 00 00

006030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006400 [0b 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00|

006410 f4 03|02|02|2e 2e 00 00 00 00 00 00 00 00 00 00

006420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

006810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

006c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

009000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

009010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

009400 00 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00

009410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

100000

在分析 superblock相应信息的时候, 对照 dumpe2fs 的信息来看, 这里给出一个样例

Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          547523b2-e0e2-413f-9982-579d7a986dc9
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              128
Block count:              1024
Reserved block count:     51
Free blocks:              986
Free inodes:              117
First block:              1
Block size:               1024
Fragment size:            1024
Reserved GDT blocks:      3
Blocks per group:         8192
Fragments per group:      8192
Inodes per group:         128
Inode blocks per group:   16
Filesystem created:       Sat May  7 16:02:36 2016
Last mount time:          Sat May  7 16:14:34 2016
Last write time:          Sat May  7 16:16:05 2016
Mount count:              2
Maximum mount count:      -1
Last checked:             Sat May  7 16:02:36 2016
Check interval:           0 (<none>)
Lifetime writes:          3 kB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          128
Default directory hash:   half_md4
Directory Hash Seed:      61eb9a73-78b3-4127-9804-5a9f4bb0752b


Group 0: (Blocks 1-1023)
  Primary superblock at 1, Group descriptors at 2-2
  Reserved GDT blocks at 3-5
  Block bitmap at 6 (+5), Inode bitmap at 7 (+6)
  Inode table at 8-23 (+7)
  986 free blocks, 117 free inodes, 2 directories
  Free blocks: 38-1023
  Free inodes: 12-128

Filesystem volume name: <none>

Last mounted on: <not available>

Filesystem UUID: 547523b2-e0e2-413f-9982-579d7a986dc9

Filesystem magic number: 0xEF53

Filesystem revision #: 1 (dynamic)

Filesystem features: ext_attr resize_inode dir_index filetype sparse_super large_file

Filesystem flags: signed_directory_hash

Default mount options: user_xattr acl

Filesystem state: clean

Errors behavior: Continue

Filesystem OS type: Linux

Inode count: 128

Block count: 1024

Reserved block count: 51

Free blocks: 986

Free inodes: 117

First block: 1

Block size: 1024

Fragment size: 1024

Reserved GDT blocks: 3

Blocks per group: 8192

Fragments per group: 8192

Inodes per group: 128

Inode blocks per group: 16

Filesystem created: Sat May 7 16:02:36 2016

Last mount time: Sat May 7 16:14:34 2016

Last write time: Sat May 7 16:16:05 2016

Mount count: 2

Maximum mount count: -1

Last checked: Sat May 7 16:02:36 2016

Check interval: 0 (<none>)

Lifetime writes: 3 kB

Reserved blocks uid: 0 (user root)

Reserved blocks gid: 0 (group root)

First inode: 11

Inode size: 128

Default directory hash: half_md4

Directory Hash Seed: 61eb9a73-78b3-4127-9804-5a9f4bb0752b

Group 0: (Blocks 1-1023)

Primary superblock at 1, Group descriptors at 2-2

Reserved GDT blocks at 3-5

Block bitmap at 6 (+5), Inode bitmap at 7 (+6)

Inode table at 8-23 (+7)

986 free blocks, 117 free inodes, 2 directories

Free blocks: 38-1023

Free inodes: 12-128

在查看相应的文件的inode, data block的信息的时候, 对照 debugfs 的信息来看 (使用 debugfs 的 stat 命令)

上面只给出了Superblock GDT, block & inode bitmap, 以及 “/”的 inode, 和 datablock的分析, 下面请试着通过上述信息分析lost+found目录的inode, datablock (参考答案在下面)

000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  # Frist Position is Boot Block
*
000400 |80 00 00 00|00 04 00 00|33 00 00 00|da 03 00 00| # Start of Super Block
inode count = 0x80(128), block count = 0x0400(1024) reserved block count = 0x33(51) free blocks = 0x03da(986)
000410 |75 00 00 00|01 00 00 00|00 00 00 00 00 00 00 00|
free inodes = 0x75(117), first block = 0x01(1), for log use(unused)
000420 |00 20 00 00|00 20 00 00|80 00 00 00|ea a3 2d 57
blocks = 0x2000(8192), fragments = 0x2000(8192), inodes/group = 0x80(128), lastmounttime = 0x572da3ea
000430 |45 a4 2d 57|02 00|ff ff|53 ef|01 00|01 00|00 00|
last write time = 0x572da445, mount count = 0x03(2), max count = 0xffff, magic = 0xEF53, fs_state = 0x01(clean), err behavior = continue, minor rev = 0
000440 |1c a1 2d 57|00 00 00 00|00 00 00 00|01 00 00 00|
last checked 0x572da11c, check interval = 0x00, os_type = 0x00(Linux), major rev = 0x01
000450 |00 00|00 00|0b 00 00 00|80 00|00 00|38 00 00 00|
reserved blocks uid = 0x00, reserved blocks gid = 0x00, first nonreserved inode = 11
(So there are 128 -11 = 117 free inodes at this time (clean and new fs)), inode size = 0x80, block group = 0, compatible features= (0x38)
000460 |02 00 00 00 03 00 00 00|54 75 23 b2 e0 e2 41 3f
compatible features, UUID = （这个不用管little endian)  547523b2-e0e2-413f-9982-579d7a986dc9
000470 99 82 57 9d 7a 98 6d c9| 00 00 00 00 00 00 00 00
UUID, continued, volumn name = NULL
000480 00 00 00 00 00 00 00 00|00 00 00 00|00|00|00 00
last mounted on = NULL, alogrithm usage bitmap, pre_alloc , prealloc, padding 
*
0004c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 00
0004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0004e0 00 00 00 00 00 00 00 00 00 00 00 00 61 eb 9a 73
0004f0 78 b3 41 27 98 04 5a 9f 4b b0 75 2b 01 00 00 00
000500 0c 00 00 00 00 00 00 00 1c a1 2d 57 00 00 00 00
000510 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
000560 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000570 00 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00
000580 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
000800 |06 00 00 00|07 00 00 00|08 00 00 00|da 03|75 00| # Group Description Table(For this sample_fs only one block to store)
block bitmap at 0x6, inode bitmap at 0x7, inode table at 0x8, freeblocks = 0x3da(986), free inodes = 0x75(117)
000810 |02 00|04 00 00 00 00 00 00 00 00 00 00 00 00 00|
directories = 0x2, padding
000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
001800 ff ff ff ff 1f 00 00 00 00 00 00 00 00 00 00 00  # Block Bitmap
001810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
001870 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80
001880 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
*
001c00 ff 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00  # This line is Inode Bitmap (128 Bit already)
ff 07 = 1111 1111 0000 0111 , That is 11 inodes (10 for fs reserve, one(inode no = 11) for lost+found directory, one
(inode no = 2) for root directory)
001c10 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
*
002000 00 00 00 00 00 00 00 00 1c a1 2d 57 1c a1 2d 57
002010 1c a1 2d 57 00 00 00 00 00 00 00 00 00 00 00 00
002020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
002080 |ed 41|00 00|00 04 00 00|40 a1 2d 57|1c a1 2d 57|  # This is the inode block for root dir "/" (Inode 2)
st_mode = 0x41ed = 040755(st_mode contains filetype & permission), user = 0, size = 1024, atime, ctime
002090 |1c a1 2d 57|00 00 00 00|00 00|03 00|02 00 00 00|
mtime, dtime, group = 0000, links = 3("/", and ".", and".."), block_count = 2(512(Sector size = 512) * 2)
0020a0 |00 00 00 00|00 00 00 00|18 00 00 00|00 00 00 00|
flags = 0x00, OS_Information, blocks[0] = 0x18(24)[This means the Block No.24 Contains data for this inode, and we can get the address for Block No.24(0x400 * 24 = 0x6000)], blocks[1] = 0
0020b0 |00 00 00 00|00 00 00 00|00 00 00 00|00 00 00 00|
blocks[2] = 0, [3], [4], [5],...

*
002300 80 81 00 00 00 30 04 04 1c a1 2d 57 1c a1 2d 57
002310 1c a1 2d 57 00 00 00 00 00 00 01 00 08 00 00 00
002320 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
002350 00 00 00 00 00 00 00 00 00 00 00 00 25 00 00 00
002360 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
002500 |c0 41|00 00|00 30 00 00|1c a1 2d 57|1c a1 2d 57|# This is the inode block for "lost+found" dir (Inode 12)
st_mode = 0x41c0(040700), user = 0, size = 0x3000(12288), atime, ctime
002510 |1c a1 2d 57|00 00 00 00|00 00|02 00|18 00 00 00|
mtime, dtime, group = 0, links = 2, block_count = 0x18(size 512 * 0x18 = 12288) (actual block = 0x18 / 2 = 0xc)
002520 |00 00 00 00|00 00 00 00|19 00 00 00|1a 00 00 00|
flags = 0x00, OS_info, Blocks[0] = 0x19(25), [1] = 0x1a,
002530 |1b 00 00 00|1c 00 00 00|1d 00 00 00|1e 00 00 00|
[2],[3],[4], ...
002540 |1f 00 00 00|20 00 00 00|21 00 00 00|22 00 00 00|
, ...
002550 |23 00 00 00|24 00 00 00|00 00 00 00 00 00 00 00
,...
002560 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006000 [|02 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00 # This is the data block for rootfs "/"( inode no 2, block no 24, type+permission 040755)
[inode = 2, record len = 12(Len(inBytes) of one record, hereis 02 00 00 00 0c 00 01 02 2e 00 00 00), namelen = 1, filetype = 2(directory), content of the file (".")], [Next has the same format as prev record, only present the data:2,12,2,2(dir),".."]
006010 |0c 00|02|02|2e 2e 00 00|][|0b 00 00 00|e8 03|0a|02
006020 6c 6f 73 74 2b 66 6f 75 6e 64 00 00 00 00 00 00
006030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006400 [0b 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00|  # This is the datablock (Block 25) for "lost+found"
[inode no = 0x0b(11), record len = 12, namelen = 1, filetype = 2(dir), content = "."]
006410 f4 03|02|02|2e 2e 00 00 00 00 00 00 00 00 00 00
[inode no = 0x02, record len = 0x0400 - 0xc = 0x03f4( This record use the whole space left in the block ), namelen = 2, filetype = 2, content = "..\0\0\0\.....\0"]
006420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00  
006810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
006c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
006c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
007c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
007c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
008c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
008c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
009000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00
009010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
009400 00 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
009410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
100000

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # Frist Position is Boot Block

000400 |80 00 00 00|00 04 00 00|33 00 00 00|da 03 00 00| # Start of Super Block

inode count = 0x80(128), block count = 0x0400(1024) reserved block count = 0x33(51) free blocks = 0x03da(986)

000410 |75 00 00 00|01 00 00 00|00 00 00 00 00 00 00 00|

free inodes = 0x75(117), first block = 0x01(1), for log use(unused)

000420 |00 20 00 00|00 20 00 00|80 00 00 00|ea a3 2d 57

blocks = 0x2000(8192), fragments = 0x2000(8192), inodes/group = 0x80(128), lastmounttime = 0x572da3ea

000430 |45 a4 2d 57|02 00|ff ff|53 ef|01 00|01 00|00 00|

last write time = 0x572da445, mount count = 0x03(2), max count = 0xffff, magic = 0xEF53, fs_state = 0x01(clean), err behavior = continue, minor rev = 0

000440 |1c a1 2d 57|00 00 00 00|00 00 00 00|01 00 00 00|

last checked 0x572da11c, check interval = 0x00, os_type = 0x00(Linux), major rev = 0x01

000450 |00 00|00 00|0b 00 00 00|80 00|00 00|38 00 00 00|

reserved blocks uid = 0x00, reserved blocks gid = 0x00, first nonreserved inode = 11

(So there are 128 -11 = 117 free inodes at this time (clean and new fs)), inode size = 0x80, block group = 0, compatible features= (0x38)

000460 |02 00 00 00 03 00 00 00|54 75 23 b2 e0 e2 41 3f

compatible features, UUID = （这个不用管little endian) 547523b2-e0e2-413f-9982-579d7a986dc9

000470 99 82 57 9d 7a 98 6d c9| 00 00 00 00 00 00 00 00

UUID, continued, volumn name = NULL

000480 00 00 00 00 00 00 00 00|00 00 00 00|00|00|00 00

last mounted on = NULL, alogrithm usage bitmap, pre_alloc , prealloc, padding

0004c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 00

0004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0004e0 00 00 00 00 00 00 00 00 00 00 00 00 61 eb 9a 73

0004f0 78 b3 41 27 98 04 5a 9f 4b b0 75 2b 01 00 00 00

000500 0c 00 00 00 00 00 00 00 1c a1 2d 57 00 00 00 00

000510 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

000560 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

000570 00 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00

000580 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

000800 |06 00 00 00|07 00 00 00|08 00 00 00|da 03|75 00| # Group Description Table(For this sample_fs only one block to store)

block bitmap at 0x6, inode bitmap at 0x7, inode table at 0x8, freeblocks = 0x3da(986), free inodes = 0x75(117)

000810 |02 00|04 00 00 00 00 00 00 00 00 00 00 00 00 00|

directories = 0x2, padding

000820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

001800 ff ff ff ff 1f 00 00 00 00 00 00 00 00 00 00 00 # Block Bitmap

001810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

001870 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80

001880 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

001c00 ff 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 # This line is Inode Bitmap (128 Bit already)

ff 07 = 1111 1111 0000 0111 , That is 11 inodes (10 for fs reserve, one(inode no = 11) for lost+found directory, one

(inode no = 2) for root directory)

001c10 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

002000 00 00 00 00 00 00 00 00 1c a1 2d 57 1c a1 2d 57

002010 1c a1 2d 57 00 00 00 00 00 00 00 00 00 00 00 00

002020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

002080 |ed 41|00 00|00 04 00 00|40 a1 2d 57|1c a1 2d 57| # This is the inode block for root dir "/" (Inode 2)

st_mode = 0x41ed = 040755(st_mode contains filetype & permission), user = 0, size = 1024, atime, ctime

002090 |1c a1 2d 57|00 00 00 00|00 00|03 00|02 00 00 00|

mtime, dtime, group = 0000, links = 3("/", and ".", and".."), block_count = 2(512(Sector size = 512) * 2)

0020a0 |00 00 00 00|00 00 00 00|18 00 00 00|00 00 00 00|

flags = 0x00, OS_Information, blocks[0] = 0x18(24)[This means the Block No.24 Contains data for this inode, and we can get the address for Block No.24(0x400 * 24 = 0x6000)], blocks[1] = 0

0020b0 |00 00 00 00|00 00 00 00|00 00 00 00|00 00 00 00|

blocks[2] = 0, [3], [4], [5],...

002300 80 81 00 00 00 30 04 04 1c a1 2d 57 1c a1 2d 57

002310 1c a1 2d 57 00 00 00 00 00 00 01 00 08 00 00 00

002320 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

002350 00 00 00 00 00 00 00 00 00 00 00 00 25 00 00 00

002360 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

002500 |c0 41|00 00|00 30 00 00|1c a1 2d 57|1c a1 2d 57|# This is the inode block for "lost+found" dir (Inode 12)

st_mode = 0x41c0(040700), user = 0, size = 0x3000(12288), atime, ctime

002510 |1c a1 2d 57|00 00 00 00|00 00|02 00|18 00 00 00|

mtime, dtime, group = 0, links = 2, block_count = 0x18(size 512 * 0x18 = 12288) (actual block = 0x18 / 2 = 0xc)

002520 |00 00 00 00|00 00 00 00|19 00 00 00|1a 00 00 00|

flags = 0x00, OS_info, Blocks[0] = 0x19(25), [1] = 0x1a,

002530 |1b 00 00 00|1c 00 00 00|1d 00 00 00|1e 00 00 00|

[2],[3],[4], ...

002540 |1f 00 00 00|20 00 00 00|21 00 00 00|22 00 00 00|

, ...

002550 |23 00 00 00|24 00 00 00|00 00 00 00 00 00 00 00

,...

002560 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006000 [|02 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00 # This is the data block for rootfs "/"( inode no 2, block no 24, type+permission 040755)

006010 |0c 00|02|02|2e 2e 00 00|][|0b 00 00 00|e8 03|0a|02

006020 6c 6f 73 74 2b 66 6f 75 6e 64 00 00 00 00 00 00

006030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006400 [0b 00 00 00|0c 00|01|02|2e 00 00 00|][02 00 00 00| # This is the datablock (Block 25) for "lost+found"

[inode no = 0x0b(11), record len = 12, namelen = 1, filetype = 2(dir), content = "."]

006410 f4 03|02|02|2e 2e 00 00 00 00 00 00 00 00 00 00

[inode no = 0x02, record len = 0x0400 - 0xc = 0x03f4( This record use the whole space left in the block ), namelen = 2, filetype = 2, content = "..\0\0\0\.....\0"]

006420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

006810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

006c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

006c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

007c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

007c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008400 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008800 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

008c00 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

008c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

009000 00 00 00 00 00 04 00 00 00 00 00 00 00 00 00 00

009010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

009400 00 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00

009410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

100000

注:这个文件系统只有一个块组 , 每个块大小为1KB, 文件系统总共1024个块, 除去Boot block还有1023 块, 这些都属于

Group 0, 其中 Block 1是超级块, 接下来的块由GDT给出, 通过GDT知道 BlockBitmap是 Block 6,Inode bitmap是block7, inode总共有128个(超级块的信息给出), 另外在读上面数据的时候,注意要用小端序来阅读(little endin)

[开新坑] Go语言学习~

五月 5, 2016 VOID001 Comments 0 Comment

[开新坑] Go语言学习~

VOID001's WOWO

✵Merry Christmas✵ Aim at being kernel hacker, now a newbie

Browsed by
月份：2016年5月

[Linux C]ELF Trick — 获取程序内的函数名及地址

五月 30, 2016 VOID001 Comments 0 Comment

NEUOJ Tech Talk #1

五月 29, 2016 VOID001 Comments 0 Comment

开始之前

颁奖环节

Tech Talk Starts

[Linux C内核] 信号

五月 27, 2016 VOID001 Comments 0 Comment

什么是信号

产生信号的条件

信号的本质

一个无法通过Ctrl + C退出的程序

信号捕捉的实现自定义信号处理函数

[go]heka插件开发

五月 25, 2016 VOID001 Comments 4 comments

开发一个Input插件

使插件能够被heka识别(注册插件到heka中)

写好Input插件的框架

为我们的Input插件实现退出功能

Draft

voidisprogramer.com 全站启用HTTPS啦~

五月 16, 2016 VOID001 Comments 4 comments

[C Linux 内核] 文件系统 #2

五月 13, 2016 VOID001 Comments 1 comment

数据块寻址

文件&目录的操作函数

VFS

文件的引用数,dup&dup2

密码保护：accelercode 回忆录 By NEU20133823(貌似你没有昵称就用这个了233)

五月 12, 2016 VOID001 Comments 要查看留言请输入您的密码。

[C Linux 内核] 文件系统(以ext2为例)

五月 8, 2016 VOID001 Comments 1 comment

ext2文件系统的总体结构

[开新坑] Go语言学习~

五月 5, 2016 VOID001 Comments 0 Comment

开始之前

颁奖环节

Tech Talk Starts

什么是信号

产生信号的条件

信号的本质

一个无法通过Ctrl + C退出的程序

信号捕捉的实现 自定义信号处理函数

开发一个Input插件

使插件能够被heka识别(注册插件到heka中)

写好Input插件的框架

为我们的Input插件实现退出功能

数据块寻址

文件&目录的操作函数

VFS

文件的引用数,dup&dup2

ext2文件系统的总体结构

信号捕捉的实现自定义信号处理函数