基于Go Packet实现网络数据包的捕获与分析
摘要
- Packet Capturing Overview
- What is Packet Capturing
- How can it be used
- What is libpcap
- Debug Tools: tcpdump & WinPcap & snoop
- What is BPF
- What is gopacket
What is Packet Capturing
Packet capture is a computer networking term for intercepting a data packet that is crossing or moving over a specific computer network.Once a packet is captured, it is stored temporarily so that it can be analyzed. The packet is inspected to help diagnose and solve network problems and determine whether network security policies are being followed.
Packet Capture Overview
How can it be used
- Development Testing & validating & Reverse engineer APP on API
- Network Administration Seeing what traffic goes on in background,Looking for malicious traffic on networkData capturing is used to identify security flaws and breaches by determining the point of intrusion.
- Troubleshooting Managed through data capturing, troubleshooting detects the occurrence of undesired events over a network and helps solve them. If the network administrator has full access to a network resource, he can access it remotely and troubleshoot any issues.
- Security defcon Wall of Sheep.Hackers can also use packet capturing techniques to steal data that is being transmitted over a network, like Stealing credentials.When data is stolen, the network administrator can retrieve the stolen or lost information easily using data capturing techniques.
- Forensics forensics for crime investigations.Whenever viruses, worms or other intrusions are detected in computers, the network administrator determines the extent of the problem. After initial analysis, she may block some segments and network traffic in order to save historical information and network data.
<!-- more -->
What is libpcap
libpcap flow involving data copy from kernel to user space.
//Compile with: gcc find_device.c -lpcap
#include <stdio.h>
#include <pcap.h>
int main(int argc, char **argv) {
char *device;
char error_buffer[PCAP_ERRBUF_SIZE];
//Find a device
device = pcap_lookupdev(error_buffer);
if (device == NULL) {
printf("Error finding device: %sn", error_buffer);
return 1;
}
printf("Network device found: %sn", device);
return 0;
}
#include <stdio.h>
#include <time.h>
#include <pcap.h>
#include <netinet/in.h>
#include <netinet/if_ether.h>
void print_packet_info(const u_char *packet, struct pcap_pkthdr packet_header);
int main(int argc, char *argv[]) {
char *device;
char error_buffer[PCAP_ERRBUF_SIZE];
pcap_t *handle;
const u_char *packet;
struct pcap_pkthdr packet_header;
int packet_count_limit = 1;
int timeout_limit = 10000; /*In milliseconds*/
device = pcap_lookupdev(error_buffer);
if (device == NULL) {
printf("Error finding device: %sn", error_buffer);
return 1;
}
/*Open device for live capture*/
handle = pcap_open_live(
device,
BUFSIZ,
packet_count_limit,
timeout_limit,
error_buffer
);
/*Attempt to capture one packet. If there is no network traffic
and the timeout is reached, it will return NULL*/
packet = pcap_next(handle, &packet_header);
if (packet == NULL) {
printf("No packet found.n");
return 2;
}
/*Our function to output some info*/
print_packet_info(packet, packet_header);
return 0;
}
void print_packet_info(const u_char *packet, struct pcap_pkthdr packet_header) {
printf("Packet capture length: %dn", packet_header.caplen);
printf("Packet total length %dn", packet_header.len);
}
Debug Tools
#Older versions of tcpdump truncate packets to 68 or 96 bytes.
#If this is the case, use -s to capture full-sized packets:
$ tcpdump -i <interface> -s 65535 -w <some-file>
# A packet capturing tool similar to TcpDump for Solaris
$ snoop -r -o arp11.snoop -q -d nxge0 -c 150000
tcpdump
tcpdump 是一个运行在命令行下的嗅探工具。它允许用户拦截和显示发送或收到过网络连接到该计算机的TCP/IP和其他数据包。它支持针对网络层、协议、主机、网络或端口的过滤,并提供and、or、not等逻辑语句来帮助你去掉无用的信息,从而使用户能够进一步找出问题的根源。可以使用BPF来限制tcpdump产生的数据包数量。
snoop
snoop uses both the network packet filter and streams buffer modules to provide efficient capture of packets from the network. Captured packets can be displayed as they are received, or saved to a file for later inspection.
promiscuous mode
抓包工具需要工作在promiscuous mode(混杂模式)(superuser), 指一台机器的网卡能够接收所有经过它的数据流,而不论其目的地址是否是它。当网卡工作在混杂模式下时,网卡将来自接口的所有数据都捕获并交给相应的驱动程序。一般在分析网络数据作为网络故障诊断手段时用到,同时这个模式也被网络黑客利用来作为网络数据窃听的入口。
BPF
Berkeley Packet Filter,缩写BPF,是类Unix系统上数据链路层的一种接口,提供原始链路层封包的收发。BPF支持“过滤”封包,这样BPF会只把“感兴趣”的封包到上层软件,可以避免从操作系统内核向用户态复制其他封包,降低抓包的CPU的负担以及所需的缓冲区空间,从而减少丢包率。BPF的过滤功能是以BPF虚拟机机器语言的解释器的形式实现的,这种语言的程序可以抓取封包数据,对封包中的数据采取算术操作,并将结果与常量或封包中的数据或结果中的测试位比较,根据比较的结果决定接受还是拒绝封包。
BPF Overview
Go Packet
Find Devices
package main
import (
"fmt"
"log"
"github.com/google/gopacket"
"github.com/google/gopacket/layers"
"github.com/google/gopacket/pcap"
)
func main() {
fmt.Println("----------Find all devices---------n ")
devices, err := pcap.FindAllDevs()
if err != nil {
log.Fatal(err)
}
// Print device information
for _, device := range devices {
for _, address := range device.Addresses {
fmt.Println("- IP address: ", address.IP)
fmt.Println("- Subnet mask: ", address.Netmask)
}
}
/*- IP address: 45.33.110.101
- Subnet mask: ffffff00
- IP address: 2600:3c01::f03c:91ff:fee5:45b6
- Subnet mask: ffffffffffffffff0000000000000000
- IP address: fe80::f03c:91ff:fee5:45b6
- Subnet mask: ffffffffffffffff0000000000000000
- IP address: 127.0.0.1
- Subnet mask: ff000000
- IP address: ::1
- Subnet mask: ffffffffffffffffffffffffffffffff
*/
Decoding Packet Layers
Capture Packet Workflow
- Getting a list of network devices
- Capturing packets from a network device
- Analyzing packet layers
- Using Berkeley Packet Filters
package main
import (
"fmt"
"log"
"net"
"github.com/google/gopacket"
"github.com/google/gopacket/layers"
"github.com/google/gopacket/pcap"
)
func main(){
handle, err := pcap.OpenLive("eth0", 65536, true, pcap.BlockForever)
if err != nil {
fmt.Printf("Error: %sn", err)
return
}
defer handle.Close()
//Create a new PacketDataSource
src := gopacket.NewPacketSource(handle, layers.LayerTypeEthernet)
//Packets returns a channel of packets
in := src.Packets()
for {
var packet gopacket.Packet
select {
//case <-stop:
//return
case packet = <-in:
arpLayer := packet.Layer(layers.LayerTypeARP)
if arpLayer == nil {
continue
}
arp := arpLayer.(*layers.ARP)
if net.HardwareAddr(arp.SourceHwAddress).String() == "abc" {
//Do something or don't
}
tcpLayer := packet.Layer(layers.LayerTypeTCP)
if tcpLayer == nil {
continue
}
tcp := tcpLayer.(*layers.TCP)
//.......
}
}
}
Creating and Sending Packets
package main
import (
"github.com/google/gopacket"
"github.com/google/gopacket/layers"
"github.com/google/gopacket/pcap"
"log"
"net"
"time"
)
var (
device string = "eth0"
snapshot_len int32 = 1024
promiscuous bool = false
err error
timeout time.Duration = 30 * time.Second
handle *pcap.Handle
buffer gopacket.SerializeBuffer
options gopacket.SerializeOptions
)
func main() {
// Open device
handle, err = pcap.OpenLive(device, snapshot_len, promiscuous, timeout)
if err != nil {log.Fatal(err) }
defer handle.Close()
// Send raw bytes over wire
rawBytes := []byte{10, 20, 30}
err = handle.WritePacketData(rawBytes)
if err != nil {
log.Fatal(err)
}
// Create a properly formed packet, just with
// empty details. Should fill out MAC addresses,
// IP addresses, etc.
buffer = gopacket.NewSerializeBuffer()
gopacket.SerializeLayers(buffer, options,
&layers.Ethernet{},
&layers.IPv4{},
&layers.TCP{},
gopacket.Payload(rawBytes),
)
outgoingPacket := buffer.Bytes()
// Send our packet
err = handle.WritePacketData(outgoingPacket)
if err != nil {
log.Fatal(err)
}
// This time lets fill out some information
ipLayer := &layers.IPv4{
SrcIP: net.IP{127, 0, 0, 1},
DstIP: net.IP{8, 8, 8, 8},
}
ethernetLayer := &layers.Ethernet{
SrcMAC: net.HardwareAddr{0xFF, 0xAA, 0xFA, 0xAA, 0xFF, 0xAA},
DstMAC: net.HardwareAddr{0xBD, 0xBD, 0xBD, 0xBD, 0xBD, 0xBD},
}
tcpLayer := &layers.TCP{
SrcPort: layers.TCPPort(4321),
DstPort: layers.TCPPort(80),
}
// And create the packet with the layers
buffer = gopacket.NewSerializeBuffer()
gopacket.SerializeLayers(buffer, options,
ethernetLayer,
ipLayer,
tcpLayer,
gopacket.Payload(rawBytes),
)
outgoingPacket = buffer.Bytes()
}
Application
- qisniff
- 新一代Ntopng网络流量监控—可视化和架构分析
- 基于网络抓包实现kubernetes中微服务的应用级监控
专题合辑:Network Engineering
- SDN 技术指南(一): 架构概览
- SDN 技术指南(二):OpenFlow
- SDN 技术指南(四):Open vSwitch
- 浅谈基于数据分析的网络态势感知
- 网络数据包的捕获与分析(libpcap、BPF及gopacket)
- 新一代Ntopng网络流量监控—可视化和架构分析
- Cyber-Security: IPv6 & Security
- AWS or Azure : 云计算平台的趋势分析|Stack Overflow,2017
- Cyber-Security|中国香港警务处网络安全与科技罪案总警司(https://segmentfault.com/a/1190000007967510)
- 添加php的memcached扩展模块
- Android TextView中显示图片
- Nginx配置中的log_format用法梳理(设置详细的日志格式)
- 分享一个刷网页PV的python小脚本
- mysql完整备份时过滤掉某些库
- Jquery 结合Json控制Select下拉框
- ExtJs学习笔记(23)-ScriptTagProxy+XTemplate+WCF跨域取数据
- Centos7.2下Jumpserver V4.0环境安装部署记录
- 利用JQuery实现更简单的Ajax跨域请求
- 运维工作中sed常规操作命令梳理
- linux下安装php的imagick扩展模块(附php升级脚本)
- 用JS + WCF打造轻量级WebPart
- Android之assets资源
- ExtJs学习笔记(24)-Drag/Drop拖动功能
- JavaScript 教程
- JavaScript 编辑工具
- JavaScript 与HTML
- JavaScript 与Java
- JavaScript 数据结构
- JavaScript 基本数据类型
- JavaScript 特殊数据类型
- JavaScript 运算符
- JavaScript typeof 运算符
- JavaScript 表达式
- JavaScript 类型转换
- JavaScript 基本语法
- JavaScript 注释
- Javascript 基本处理流程
- Javascript 选择结构
- Javascript if 语句
- Javascript if 语句的嵌套
- Javascript switch 语句
- Javascript 循环结构
- Javascript 循环结构实例
- Javascript 跳转语句
- Javascript 控制语句总结
- Javascript 函数介绍
- Javascript 函数的定义
- Javascript 函数调用
- Javascript 几种特殊的函数
- JavaScript 内置函数简介
- Javascript eval() 函数
- Javascript isFinite() 函数
- Javascript isNaN() 函数
- parseInt() 与 parseFloat()
- escape() 与 unescape()
- Javascript 字符串介绍
- Javascript length属性
- javascript 字符串函数
- Javascript 日期对象简介
- Javascript 日期对象用途
- Date 对象属性和方法
- Javascript 数组是什么
- Javascript 创建数组
- Javascript 数组赋值与取值
- Javascript 数组属性和方法
- 实验7 3D机器人
- EasyValidate优雅地校验提交数据完整性
- 实验8 OpenGL太阳系动画
- Django实现列表页商品数据返回教程
- 实验9 OpenGL光照
- Python实现多线程下载脚本的示例代码
- Android自定义控件仿iOS滑块SwitchButton
- kotlin gson反序列化默认值失效深入讲解
- Android使用Volley实现上传文件功能
- 工作中使用jasmine遇到的一个html element和Component绑定属性失去同步的问题
- Android使用Volley框架定制PostUploadRequest上传文件
- Android实现横向滑动卡片效果
- 在jasmine beforeEach里修改UI元素的一个side effect
- Android实现头像上传功能
- 关于jasmine里debugElement.query和fixture.detectChanges的依赖关系