初始提交
This commit is contained in:
@@ -0,0 +1,67 @@
|
||||
# UDP大数据包发送模块
|
||||
|
||||
## 功能描述
|
||||
|
||||
`udpSendBigPacket` 函数用于将大数据包(最大60KB)拆分成多个小包发送。每个小包的有效载荷最大为1024字节,包含包头信息。
|
||||
|
||||
## 包头结构
|
||||
|
||||
每个发送的包都包含以下包头信息:
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
uint16_t magic; // 魔数 (0xAA55)
|
||||
uint16_t total_size; // 总数据大小
|
||||
uint32_t big_packet_id; // 大包ID,32位随机整数,用于区分不同的大包
|
||||
uint16_t packet_id; // 包序号 (0, 1, 2, ...)
|
||||
uint16_t total_packets; // 总包数
|
||||
uint16_t payload_len; // 当前包的有效载荷长度
|
||||
uint8_t reserved[2]; // 保留字段
|
||||
} udp_packet_header_t;
|
||||
```
|
||||
|
||||
## 使用示例
|
||||
|
||||
```c
|
||||
#include "udp.h"
|
||||
|
||||
// 准备要发送的数据
|
||||
uint8_t data[5000]; // 5KB数据
|
||||
// ... 填充数据 ...
|
||||
|
||||
// 发送大数据包
|
||||
int result = udpSendBigPacket(data, 5000, "192.168.1.100", 8080);
|
||||
if (result >= 0) {
|
||||
printf("发送成功\n");
|
||||
} else {
|
||||
printf("发送失败,错误码: %d\n", result);
|
||||
}
|
||||
```
|
||||
|
||||
## 错误码说明
|
||||
|
||||
- `0`: 发送成功
|
||||
- `-1`: 参数错误(空指针、长度超出范围等)
|
||||
- `-2`: 内存分配失败
|
||||
- 其他负值: `udpSendTo` 函数返回的错误码
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. 函数会自动将数据拆分成多个包发送,无需手动处理
|
||||
2. 每个包都会包含完整的包头信息
|
||||
3. 第一包会指定总数据大小,便于接收端重组
|
||||
4. 函数不等待回复,采用异步发送方式
|
||||
5. 最大支持60KB的数据包
|
||||
6. 每个包的有效载荷最大1024字节
|
||||
7. 每个大包都有唯一的32位随机ID,用于区分并发传输的不同大包
|
||||
8. 大包ID使用 `rand()` 函数生成,确保唯一性
|
||||
|
||||
## 接收端处理
|
||||
|
||||
接收端需要:
|
||||
1. 检查魔数 (0xAA55)
|
||||
2. 根据 `big_packet_id` 区分不同的大包
|
||||
3. 根据 `total_size` 和 `total_packets` 分配缓冲区
|
||||
4. 按 `packet_id` 顺序重组数据
|
||||
5. 验证所有包是否接收完整
|
||||
6. 使用 `big_packet_id` 可以同时处理多个并发的大包传输
|
||||
@@ -0,0 +1,164 @@
|
||||
# UDP大数据包接收端 (Python实现)
|
||||
|
||||
## 功能特性
|
||||
|
||||
- **多发送方支持**: 可以同时处理多个发送方的并发传输
|
||||
- **超时机制**: 1秒超时,超时的包自动丢弃
|
||||
- **包重组**: 自动将分散的小包重组成完整数据
|
||||
- **线程安全**: 使用锁机制确保多线程环境下的数据安全
|
||||
- **实时日志**: 详细的日志输出,便于调试和监控
|
||||
|
||||
## 安装和运行
|
||||
|
||||
### 基本运行
|
||||
|
||||
```bash
|
||||
# 启动接收端(默认监听8080端口)
|
||||
python udp_receiver.py
|
||||
|
||||
# 指定端口
|
||||
python udp_receiver.py --port 9000
|
||||
|
||||
# 指定监听地址
|
||||
python udp_receiver.py --host 192.168.1.100 --port 8080
|
||||
|
||||
# 启用调试日志
|
||||
python udp_receiver.py --debug
|
||||
```
|
||||
|
||||
### 参数说明
|
||||
|
||||
- `--host`: 监听地址 (默认: 0.0.0.0)
|
||||
- `--port`: 监听端口 (默认: 8080)
|
||||
- `--timeout`: 包超时时间,秒 (默认: 1.0)
|
||||
- `--debug`: 启用调试日志
|
||||
|
||||
## 测试
|
||||
|
||||
### 使用测试发送端
|
||||
|
||||
```bash
|
||||
# 测试单个发送端
|
||||
python test_sender.py --test single
|
||||
|
||||
# 测试多个发送端同时发送
|
||||
python test_sender.py --test multiple
|
||||
|
||||
# 测试超时场景
|
||||
python test_sender.py --test timeout
|
||||
```
|
||||
|
||||
### 手动测试
|
||||
|
||||
```bash
|
||||
# 终端1: 启动接收端
|
||||
python udp_receiver.py --debug
|
||||
|
||||
# 终端2: 启动发送端
|
||||
python test_sender.py --test single
|
||||
```
|
||||
|
||||
## 核心类说明
|
||||
|
||||
### UDPPacketHeader
|
||||
包头解析类,与C端保持一致的包头结构:
|
||||
```python
|
||||
class UDPPacketHeader:
|
||||
MAGIC = 0xAA55
|
||||
HEADER_SIZE = 16
|
||||
|
||||
# 字段: magic, total_size, big_packet_id, packet_id, total_packets, payload_len
|
||||
```
|
||||
|
||||
### PacketReassembler
|
||||
包重组器,负责单个大包的重组:
|
||||
- 管理包的重组状态
|
||||
- 检查超时
|
||||
- 按序号重组数据
|
||||
|
||||
### UDPReceiver
|
||||
主接收端类:
|
||||
- 监听UDP端口
|
||||
- 管理多个重组器
|
||||
- 处理并发传输
|
||||
- 自动清理超时包
|
||||
|
||||
## 工作流程
|
||||
|
||||
1. **接收包**: 监听UDP端口,接收数据包
|
||||
2. **解析包头**: 验证魔数,解析包头信息
|
||||
3. **创建/获取重组器**: 根据发送方地址和大包ID创建或获取重组器
|
||||
4. **添加包**: 将包添加到对应的重组器
|
||||
5. **检查完成**: 检查是否所有包都已接收
|
||||
6. **重组数据**: 按序号重组完整数据
|
||||
7. **处理数据**: 保存或处理重组后的数据
|
||||
8. **清理**: 自动清理超时的包
|
||||
|
||||
## 超时机制
|
||||
|
||||
- **超时时间**: 1秒(可配置)
|
||||
- **检查频率**: 每0.5秒检查一次
|
||||
- **清理策略**: 超时的包自动丢弃,释放内存
|
||||
- **日志记录**: 超时事件会记录到日志
|
||||
|
||||
## 并发处理
|
||||
|
||||
- **多发送方**: 支持多个发送方同时发送
|
||||
- **大包ID**: 使用32位随机ID区分不同的大包
|
||||
- **线程安全**: 使用锁机制保护共享数据
|
||||
- **独立重组**: 每个大包有独立的重组器
|
||||
|
||||
## 日志输出
|
||||
|
||||
### 信息级别
|
||||
- `INFO`: 重要事件(启动、完成、创建重组器)
|
||||
- `WARNING`: 警告事件(超时、错误)
|
||||
- `ERROR`: 错误事件(解析失败、网络错误)
|
||||
- `DEBUG`: 调试信息(包接收详情)
|
||||
|
||||
### 日志示例
|
||||
```
|
||||
2024-01-01 12:00:00,000 - INFO - UDP接收端启动,监听 0.0.0.0:8080
|
||||
2024-01-01 12:00:01,123 - INFO - 创建新的大包重组器: ID=12345, 大小=5000, 包数=5, 来源=('192.168.1.100', 12345)
|
||||
2024-01-01 12:00:01,234 - DEBUG - 接收包 1/5 来自 ('192.168.1.100', 12345), 大包ID: 12345
|
||||
2024-01-01 12:00:01,345 - INFO - 大包 12345 重组完成,大小: 5000 字节
|
||||
2024-01-01 12:00:01,456 - INFO - 收到完整数据包: 大小=5000 字节, 来源=('192.168.1.100', 12345), ID=12345
|
||||
2024-01-01 12:00:02,000 - WARNING - 清理超时的大包: ID=67890, 来源=('192.168.1.101', 54321), 已接收=2/5
|
||||
```
|
||||
|
||||
## 数据保存
|
||||
|
||||
默认情况下,接收到的完整数据会保存为二进制文件:
|
||||
- 文件名格式: `received_data_{big_packet_id}_{timestamp}.bin`
|
||||
- 可以修改 `_handle_completed_packet` 方法来自定义数据处理逻辑
|
||||
|
||||
## 性能考虑
|
||||
|
||||
- **内存使用**: 每个重组器占用少量内存,超时后自动释放
|
||||
- **CPU使用**: 轻量级处理,适合高并发场景
|
||||
- **网络效率**: 支持大包传输,减少网络开销
|
||||
- **扩展性**: 模块化设计,易于扩展功能
|
||||
|
||||
## 错误处理
|
||||
|
||||
- **网络错误**: 自动重试,记录错误日志
|
||||
- **数据错误**: 验证包头,丢弃无效数据
|
||||
- **超时处理**: 自动清理,防止内存泄漏
|
||||
- **异常恢复**: 优雅处理异常,保持服务稳定
|
||||
|
||||
## 与C端兼容性
|
||||
|
||||
Python接收端完全兼容C端发送的包格式:
|
||||
- 相同的包头结构
|
||||
- 相同的魔数 (0xAA55)
|
||||
- 相同的字节序 (小端序)
|
||||
- 相同的字段定义
|
||||
|
||||
## 扩展建议
|
||||
|
||||
1. **数据验证**: 添加CRC校验
|
||||
2. **压缩支持**: 支持数据压缩
|
||||
3. **加密传输**: 添加数据加密
|
||||
4. **统计监控**: 添加传输统计
|
||||
5. **配置管理**: 支持配置文件
|
||||
6. **API接口**: 提供REST API接口
|
||||
@@ -0,0 +1,205 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
UDP大数据包发送端测试程序
|
||||
用于测试接收端的功能
|
||||
"""
|
||||
|
||||
import socket
|
||||
import struct
|
||||
import random
|
||||
import time
|
||||
import threading
|
||||
from typing import List
|
||||
|
||||
# 包头结构定义 (与C端保持一致)
|
||||
class UDPPacketHeader:
|
||||
"""UDP包头结构"""
|
||||
MAGIC = 0xAA55
|
||||
HEADER_SIZE = 16
|
||||
|
||||
def __init__(self, magic: int, total_size: int, big_packet_id: int,
|
||||
packet_id: int, total_packets: int, payload_len: int):
|
||||
self.magic = magic
|
||||
self.total_size = total_size
|
||||
self.big_packet_id = big_packet_id
|
||||
self.packet_id = packet_id
|
||||
self.total_packets = total_packets
|
||||
self.payload_len = payload_len
|
||||
|
||||
def to_bytes(self) -> bytes:
|
||||
"""转换为字节数据"""
|
||||
return struct.pack('<HHIHHH',
|
||||
self.magic, self.total_size, self.big_packet_id,
|
||||
self.packet_id, self.total_packets, self.payload_len)
|
||||
|
||||
class UDPSender:
|
||||
"""UDP发送端"""
|
||||
|
||||
def __init__(self, target_host: str, target_port: int):
|
||||
self.target_host = target_host
|
||||
self.target_port = target_port
|
||||
self.socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
|
||||
def send_big_packet(self, data: bytes, delay: float = 0.0) -> int:
|
||||
"""
|
||||
发送大数据包
|
||||
"""
|
||||
if len(data) > 60 * 1024: # 60KB限制
|
||||
raise ValueError("数据太大,超过60KB限制")
|
||||
|
||||
# 生成随机大包ID
|
||||
big_packet_id = random.randint(1, 0xFFFFFFFF)
|
||||
|
||||
# 计算包数量
|
||||
max_payload_size = 1024
|
||||
total_packets = (len(data) + max_payload_size - 1) // max_payload_size
|
||||
|
||||
print(f"发送大包: ID={big_packet_id}, 大小={len(data)} 字节, 包数={total_packets}")
|
||||
|
||||
# 发送所有包
|
||||
for packet_id in range(total_packets):
|
||||
# 计算当前包的载荷
|
||||
start_offset = packet_id * max_payload_size
|
||||
end_offset = min(start_offset + max_payload_size, len(data))
|
||||
payload = data[start_offset:end_offset]
|
||||
payload_len = len(payload)
|
||||
|
||||
# 创建包头
|
||||
header = UDPPacketHeader(
|
||||
magic=UDPPacketHeader.MAGIC,
|
||||
total_size=len(data),
|
||||
big_packet_id=big_packet_id,
|
||||
packet_id=packet_id,
|
||||
total_packets=total_packets,
|
||||
payload_len=payload_len
|
||||
)
|
||||
|
||||
# 组装完整包
|
||||
packet = header.to_bytes() + payload
|
||||
|
||||
# 发送包
|
||||
self.socket.sendto(packet, (self.target_host, self.target_port))
|
||||
|
||||
print(f" 发送包 {packet_id + 1}/{total_packets}, 载荷大小: {payload_len}")
|
||||
|
||||
# 延迟(模拟网络延迟)
|
||||
if delay > 0:
|
||||
time.sleep(delay)
|
||||
|
||||
return big_packet_id
|
||||
|
||||
def close(self):
|
||||
"""关闭socket"""
|
||||
self.socket.close()
|
||||
|
||||
def generate_test_data(size: int) -> bytes:
|
||||
"""生成测试数据"""
|
||||
return bytes([random.randint(0, 255) for _ in range(size)])
|
||||
|
||||
def test_single_sender():
|
||||
"""测试单个发送端"""
|
||||
sender = UDPSender('127.0.0.1', 8080)
|
||||
|
||||
try:
|
||||
# 发送不同大小的数据包
|
||||
test_sizes = [100, 1000, 5000, 10000, 30000]
|
||||
|
||||
for size in test_sizes:
|
||||
print(f"\n=== 测试发送 {size} 字节数据 ===")
|
||||
data = generate_test_data(size)
|
||||
big_packet_id = sender.send_big_packet(data, delay=0.01)
|
||||
print(f"大包 {big_packet_id} 发送完成")
|
||||
time.sleep(0.5) # 等待接收端处理
|
||||
|
||||
finally:
|
||||
sender.close()
|
||||
|
||||
def test_multiple_senders():
|
||||
"""测试多个发送端同时发送"""
|
||||
def sender_thread(sender_id: int, data_size: int):
|
||||
sender = UDPSender('127.0.0.1', 8080)
|
||||
try:
|
||||
data = generate_test_data(data_size)
|
||||
big_packet_id = sender.send_big_packet(data, delay=0.02)
|
||||
print(f"发送端 {sender_id}: 大包 {big_packet_id} 发送完成")
|
||||
finally:
|
||||
sender.close()
|
||||
|
||||
# 创建多个发送线程
|
||||
threads = []
|
||||
for i in range(3):
|
||||
data_size = random.randint(1000, 10000)
|
||||
thread = threading.Thread(target=sender_thread, args=(i, data_size))
|
||||
threads.append(thread)
|
||||
thread.start()
|
||||
|
||||
# 等待所有线程完成
|
||||
for thread in threads:
|
||||
thread.join()
|
||||
|
||||
print("所有发送端完成")
|
||||
|
||||
def test_timeout_scenario():
|
||||
"""测试超时场景(故意延迟发送)"""
|
||||
sender = UDPSender('127.0.0.1', 8080)
|
||||
|
||||
try:
|
||||
print("=== 测试超时场景 ===")
|
||||
data = generate_test_data(5000)
|
||||
|
||||
# 只发送前几个包,然后等待超时
|
||||
big_packet_id = random.randint(1, 0xFFFFFFFF)
|
||||
total_packets = (len(data) + 1024 - 1) // 1024
|
||||
|
||||
for packet_id in range(min(3, total_packets)): # 只发送前3个包
|
||||
start_offset = packet_id * 1024
|
||||
end_offset = min(start_offset + 1024, len(data))
|
||||
payload = data[start_offset:end_offset]
|
||||
|
||||
header = UDPPacketHeader(
|
||||
magic=UDPPacketHeader.MAGIC,
|
||||
total_size=len(data),
|
||||
big_packet_id=big_packet_id,
|
||||
packet_id=packet_id,
|
||||
total_packets=total_packets,
|
||||
payload_len=len(payload)
|
||||
)
|
||||
|
||||
packet = header.to_bytes() + payload
|
||||
sender.socket.sendto(packet, (sender.target_host, sender.target_port))
|
||||
print(f"发送包 {packet_id + 1}/{total_packets} (故意不发送完整)")
|
||||
|
||||
print("等待超时...")
|
||||
time.sleep(2) # 等待超时
|
||||
|
||||
finally:
|
||||
sender.close()
|
||||
|
||||
def main():
|
||||
"""主函数"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description='UDP大数据包发送端测试程序')
|
||||
parser.add_argument('--host', default='127.0.0.1', help='目标主机 (默认: 127.0.0.1)')
|
||||
parser.add_argument('--port', type=int, default=8080, help='目标端口 (默认: 8080)')
|
||||
parser.add_argument('--test', choices=['single', 'multiple', 'timeout'],
|
||||
default='single', help='测试类型 (默认: single)')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
print(f"UDP发送端测试程序")
|
||||
print(f"目标: {args.host}:{args.port}")
|
||||
print(f"测试类型: {args.test}")
|
||||
|
||||
if args.test == 'single':
|
||||
test_single_sender()
|
||||
elif args.test == 'multiple':
|
||||
test_multiple_senders()
|
||||
elif args.test == 'timeout':
|
||||
test_timeout_scenario()
|
||||
|
||||
print("测试完成")
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,313 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
UDP大数据包接收端
|
||||
支持多个发送方同时发送,包含1秒超时机制
|
||||
"""
|
||||
|
||||
import socket
|
||||
import struct
|
||||
import threading
|
||||
import time
|
||||
from collections import defaultdict
|
||||
from typing import Dict, List, Tuple, Optional
|
||||
import logging
|
||||
|
||||
# 配置日志
|
||||
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# 包头结构定义 (与C端保持一致)
|
||||
class UDPPacketHeader:
|
||||
"""UDP包头结构"""
|
||||
MAGIC = 0xAA55
|
||||
HEADER_SIZE = 16 # 包头大小
|
||||
|
||||
def __init__(self, data: bytes):
|
||||
"""
|
||||
从字节数据解析包头
|
||||
"""
|
||||
if len(data) < self.HEADER_SIZE:
|
||||
raise ValueError("数据长度不足,无法解析包头")
|
||||
|
||||
# 解析包头字段 (小端序)
|
||||
(self.magic, self.total_size, self.big_packet_id,
|
||||
self.packet_id, self.total_packets, self.payload_len) = struct.unpack('<HHIHHH', data[:16])
|
||||
|
||||
# 验证魔数
|
||||
if self.magic != self.MAGIC:
|
||||
raise ValueError(f"无效的魔数: {self.magic:04X}, 期望: {self.MAGIC:04X}")
|
||||
|
||||
class PacketReassembler:
|
||||
"""包重组器,处理单个大包的重组"""
|
||||
|
||||
def __init__(self, big_packet_id: int, total_size: int, total_packets: int,
|
||||
sender_addr: Tuple[str, int], timeout: float = 1.0):
|
||||
self.big_packet_id = big_packet_id
|
||||
self.total_size = total_size
|
||||
self.total_packets = total_packets
|
||||
self.sender_addr = sender_addr
|
||||
self.timeout = timeout
|
||||
self.start_time = time.time()
|
||||
|
||||
# 存储接收到的包
|
||||
self.packets: Dict[int, bytes] = {}
|
||||
self.received_count = 0
|
||||
self.completed = False
|
||||
self.data = None
|
||||
|
||||
def add_packet(self, packet_id: int, payload: bytes) -> bool:
|
||||
"""
|
||||
添加一个包到重组器
|
||||
返回True如果包是新接收的,False如果包已存在
|
||||
"""
|
||||
if packet_id in self.packets:
|
||||
return False # 包已存在
|
||||
|
||||
self.packets[packet_id] = payload
|
||||
self.received_count += 1
|
||||
|
||||
# 检查是否所有包都已接收
|
||||
if self.received_count == self.total_packets:
|
||||
self._reassemble()
|
||||
return True
|
||||
|
||||
return True
|
||||
|
||||
def _reassemble(self):
|
||||
"""重组完整数据"""
|
||||
try:
|
||||
# 按包序号排序
|
||||
sorted_packets = sorted(self.packets.items())
|
||||
|
||||
# 重组数据
|
||||
reassembled_data = b''
|
||||
for packet_id, payload in sorted_packets:
|
||||
reassembled_data += payload
|
||||
|
||||
# 截取到指定长度
|
||||
self.data = reassembled_data[:self.total_size]
|
||||
self.completed = True
|
||||
|
||||
logger.info(f"大包 {self.big_packet_id} 重组完成,大小: {len(self.data)} 字节")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"重组大包 {self.big_packet_id} 时出错: {e}")
|
||||
|
||||
def is_timeout(self) -> bool:
|
||||
"""检查是否超时"""
|
||||
return time.time() - self.start_time > self.timeout
|
||||
|
||||
def is_completed(self) -> bool:
|
||||
"""检查是否完成重组"""
|
||||
return self.completed
|
||||
|
||||
def get_data(self) -> Optional[bytes]:
|
||||
"""获取重组后的数据"""
|
||||
return self.data if self.completed else None
|
||||
|
||||
class UDPReceiver:
|
||||
"""UDP接收端主类"""
|
||||
|
||||
def __init__(self, host: str = '0.0.0.0', port: int = 8080, timeout: float = 1.0):
|
||||
self.host = host
|
||||
self.port = port
|
||||
self.timeout = timeout
|
||||
self.socket = None
|
||||
self.running = False
|
||||
|
||||
# 存储所有正在重组的大包
|
||||
# key: (sender_addr, big_packet_id), value: PacketReassembler
|
||||
self.reassemblers: Dict[Tuple[Tuple[str, int], int], PacketReassembler] = {}
|
||||
self.reassembler_lock = threading.Lock()
|
||||
|
||||
# 清理线程
|
||||
self.cleanup_thread = None
|
||||
|
||||
def start(self):
|
||||
"""启动接收端"""
|
||||
try:
|
||||
self.socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
self.socket.bind((self.host, self.port))
|
||||
self.socket.settimeout(0.1) # 设置socket超时,用于检查停止信号
|
||||
|
||||
self.running = True
|
||||
logger.info(f"UDP接收端启动,监听 {self.host}:{self.port}")
|
||||
|
||||
# 启动清理线程
|
||||
self.cleanup_thread = threading.Thread(target=self._cleanup_timeout_packets, daemon=True)
|
||||
self.cleanup_thread.start()
|
||||
|
||||
# 主接收循环
|
||||
self._receive_loop()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"启动接收端失败: {e}")
|
||||
self.stop()
|
||||
|
||||
def stop(self):
|
||||
"""停止接收端"""
|
||||
self.running = False
|
||||
if self.socket:
|
||||
self.socket.close()
|
||||
logger.info("UDP接收端已停止")
|
||||
|
||||
def _receive_loop(self):
|
||||
"""主接收循环"""
|
||||
while self.running:
|
||||
try:
|
||||
data, addr = self.socket.recvfrom(2048) # 最大接收2048字节
|
||||
self._process_packet(data, addr)
|
||||
|
||||
except socket.timeout:
|
||||
continue # 超时继续循环
|
||||
except Exception as e:
|
||||
if self.running: # 只有在运行状态下才记录错误
|
||||
logger.error(f"接收数据时出错: {e}")
|
||||
|
||||
def _process_packet(self, data: bytes, addr: Tuple[str, int]):
|
||||
"""处理接收到的数据包"""
|
||||
try:
|
||||
# 解析包头
|
||||
header = UDPPacketHeader(data)
|
||||
|
||||
# 提取载荷数据
|
||||
payload = data[UDPPacketHeader.HEADER_SIZE:UDPPacketHeader.HEADER_SIZE + header.payload_len]
|
||||
|
||||
# 获取或创建重组器
|
||||
reassembler = self._get_or_create_reassembler(header, addr)
|
||||
|
||||
if reassembler is None:
|
||||
return # 超时或无效的包
|
||||
|
||||
# 添加包到重组器
|
||||
if reassembler.add_packet(header.packet_id, payload):
|
||||
logger.debug(f"接收包 {header.packet_id}/{header.total_packets} "
|
||||
f"来自 {addr}, 大包ID: {header.big_packet_id}")
|
||||
|
||||
# 检查是否完成重组
|
||||
if reassembler.is_completed():
|
||||
self._handle_completed_packet(reassembler)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"处理来自 {addr} 的数据包时出错: {e}")
|
||||
|
||||
def _get_or_create_reassembler(self, header: UDPPacketHeader, addr: Tuple[str, int]) -> Optional[PacketReassembler]:
|
||||
"""获取或创建包重组器"""
|
||||
key = (addr, header.big_packet_id)
|
||||
|
||||
with self.reassembler_lock:
|
||||
# 检查是否已存在
|
||||
if key in self.reassemblers:
|
||||
reassembler = self.reassemblers[key]
|
||||
|
||||
# 检查是否超时
|
||||
if reassembler.is_timeout():
|
||||
logger.warning(f"大包 {header.big_packet_id} 超时,丢弃")
|
||||
del self.reassemblers[key]
|
||||
return None
|
||||
|
||||
return reassembler
|
||||
|
||||
# 创建新的重组器
|
||||
reassembler = PacketReassembler(
|
||||
header.big_packet_id,
|
||||
header.total_size,
|
||||
header.total_packets,
|
||||
addr,
|
||||
self.timeout
|
||||
)
|
||||
|
||||
self.reassemblers[key] = reassembler
|
||||
logger.info(f"创建新的大包重组器: ID={header.big_packet_id}, "
|
||||
f"大小={header.total_size}, 包数={header.total_packets}, 来源={addr}")
|
||||
|
||||
return reassembler
|
||||
|
||||
def _handle_completed_packet(self, reassembler: PacketReassembler):
|
||||
"""处理完成重组的数据包"""
|
||||
data = reassembler.get_data()
|
||||
if data:
|
||||
# 这里可以添加自定义的数据处理逻辑
|
||||
logger.info(f"收到完整数据包: 大小={len(data)} 字节, "
|
||||
f"来源={reassembler.sender_addr}, ID={reassembler.big_packet_id}")
|
||||
|
||||
# 示例:保存到文件
|
||||
filename = f"received_data_{reassembler.big_packet_id}_{int(time.time())}.bin"
|
||||
with open(filename, 'wb') as f:
|
||||
f.write(data)
|
||||
logger.info(f"数据已保存到文件: {filename}")
|
||||
|
||||
# 从重组器列表中移除
|
||||
with self.reassembler_lock:
|
||||
key = (reassembler.sender_addr, reassembler.big_packet_id)
|
||||
if key in self.reassemblers:
|
||||
del self.reassemblers[key]
|
||||
|
||||
def _cleanup_timeout_packets(self):
|
||||
"""清理超时的包(在后台线程中运行)"""
|
||||
while self.running:
|
||||
time.sleep(0.5) # 每0.5秒检查一次
|
||||
|
||||
with self.reassembler_lock:
|
||||
timeout_keys = []
|
||||
for key, reassembler in self.reassemblers.items():
|
||||
if reassembler.is_timeout():
|
||||
timeout_keys.append(key)
|
||||
|
||||
# 移除超时的包
|
||||
for key in timeout_keys:
|
||||
reassembler = self.reassemblers[key]
|
||||
logger.warning(f"清理超时的大包: ID={reassembler.big_packet_id}, "
|
||||
f"来源={reassembler.sender_addr}, "
|
||||
f"已接收={reassembler.received_count}/{reassembler.total_packets}")
|
||||
del self.reassemblers[key]
|
||||
|
||||
def get_stats(self) -> Dict:
|
||||
"""获取统计信息"""
|
||||
with self.reassembler_lock:
|
||||
return {
|
||||
'active_reassemblers': len(self.reassemblers),
|
||||
'reassemblers': [
|
||||
{
|
||||
'big_packet_id': r.big_packet_id,
|
||||
'sender': r.sender_addr,
|
||||
'received': r.received_count,
|
||||
'total': r.total_packets,
|
||||
'progress': f"{r.received_count}/{r.total_packets}",
|
||||
'time_elapsed': time.time() - r.start_time
|
||||
}
|
||||
for r in self.reassemblers.values()
|
||||
]
|
||||
}
|
||||
|
||||
def main():
|
||||
"""主函数"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description='UDP大数据包接收端')
|
||||
parser.add_argument('--host', default='0.0.0.0', help='监听地址 (默认: 0.0.0.0)')
|
||||
parser.add_argument('--port', type=int, default=8080, help='监听端口 (默认: 8080)')
|
||||
parser.add_argument('--timeout', type=float, default=1.0, help='包超时时间,秒 (默认: 1.0)')
|
||||
parser.add_argument('--debug', action='store_true', help='启用调试日志')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.debug:
|
||||
logging.getLogger().setLevel(logging.DEBUG)
|
||||
|
||||
# 创建并启动接收端
|
||||
receiver = UDPReceiver(args.host, args.port, args.timeout)
|
||||
|
||||
try:
|
||||
# 启动接收端
|
||||
receiver.start()
|
||||
except KeyboardInterrupt:
|
||||
logger.info("收到中断信号,正在停止...")
|
||||
receiver.stop()
|
||||
except Exception as e:
|
||||
logger.error(f"运行时出错: {e}")
|
||||
receiver.stop()
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,253 @@
|
||||
import socket
|
||||
import base64
|
||||
import json
|
||||
import time
|
||||
import psutil
|
||||
import struct
|
||||
import csv
|
||||
from config import *
|
||||
from x_ray_data_processor import processor
|
||||
from PySide6.QtCore import QObject, Signal, Slot, QThread,QTimer
|
||||
# 定义adcDataUnitDef结构体
|
||||
class adcDataUnitDef:
|
||||
def __init__(self):
|
||||
self.ucCmd = 0
|
||||
self.ucAddr = 0
|
||||
self.usData = 0
|
||||
|
||||
class UdpServer(QThread):
|
||||
start_cal = Signal(int, bytearray, str, int) # 信号:接收到消息,包含数据、发送者地址和端口
|
||||
send_collect_data = Signal(dict) #
|
||||
update_acquisition_info = Signal(int, dict)
|
||||
def __init__(self, ip="0.0.0.0", port=50009, parent=None):
|
||||
super(UdpServer, self).__init__(parent)
|
||||
self.kill_udp_port(port)
|
||||
self.ip = ip
|
||||
self.port = port
|
||||
self.server_socket = None
|
||||
self.running = True
|
||||
self.motor1_direction = ""
|
||||
self.motor2_direction = ""
|
||||
self.collect_object1 = {
|
||||
"method" : "scanner1",
|
||||
"channelA_data" :"base64",
|
||||
"channelB_data" :"base64",
|
||||
"intensityA" : 98,
|
||||
"intensityB" : 98
|
||||
}
|
||||
self.collect_object2 = {
|
||||
"method" : "scanner2",
|
||||
"channelA_data" :"base64",
|
||||
"channelB_data" :"base64",
|
||||
"intensityA" : 98,
|
||||
"intensityB" : 98
|
||||
}
|
||||
self.debug_timer = QTimer(self)
|
||||
self.debug_timer.timeout.connect(self.update_debug_timer)
|
||||
if YSMODE:
|
||||
self.debug_timer.start(1000)
|
||||
|
||||
def update_debug_timer(self):
|
||||
import random
|
||||
channelA = [random.randint(1, 1000) for _ in range(3000)]
|
||||
channelB = [random.randint(1, 1000) for _ in range(3000)]
|
||||
self.collect_object1['channelA_data'] = base64.b64encode(json.dumps(channelA).encode()).decode()
|
||||
self.collect_object1['channelB_data'] = base64.b64encode(json.dumps(channelB).encode()).decode()
|
||||
self.collect_object2['channelA_data'] = base64.b64encode(json.dumps(channelA).encode()).decode()
|
||||
self.collect_object2['channelB_data'] = base64.b64encode(json.dumps(channelB).encode()).decode()
|
||||
self.send_collect_data.emit(self.collect_object1)
|
||||
self.send_collect_data.emit(self.collect_object2)
|
||||
|
||||
def kill_udp_port(self, port):
|
||||
for proc in psutil.process_iter():
|
||||
try:
|
||||
for conn in proc.connections(kind='udp'):
|
||||
if conn.laddr.port == port:
|
||||
print(f" PID={proc.pid}, procname={proc.name()}")
|
||||
proc.kill()
|
||||
except psutil.AccessDenied:
|
||||
continue
|
||||
def packet_crc(self, data):
|
||||
usSum = 0
|
||||
for byte in data:
|
||||
usSum += byte
|
||||
return usSum & 0xFFFF # 确保返回值为16位
|
||||
|
||||
def handle_sample_data(self, data):
|
||||
# 确保数据长度是4的倍数
|
||||
if len(data) % 4 != 0:
|
||||
raise ValueError("Data length must be a multiple of 4")
|
||||
|
||||
# 使用列表推导式来简化代码
|
||||
unpacked_data = [struct.unpack_from('<H', data[i:i+2])[0] for i in range(0, len(data), 2)]
|
||||
|
||||
# 将数据分为两组
|
||||
data_list1 = unpacked_data[::2] # 取偶数索引的元素
|
||||
data_list2 = unpacked_data[1::2] # 取奇数索引的元素
|
||||
|
||||
# 打印数据长度,用于调试
|
||||
print(f"Data contains {len(unpacked_data) // 2} pairs of values")
|
||||
# print(data_list1)
|
||||
# print("=======================================")
|
||||
# print(data_list2)
|
||||
file_content = bytearray()
|
||||
datalen = len(data_list2)
|
||||
for i in range (datalen):
|
||||
file_content.extend(struct.pack('<ff', data_list1[i], data_list2[i]))
|
||||
return data_list1,data_list2,file_content,datalen
|
||||
|
||||
# @Slot(int, str)
|
||||
# def update_direction(self, motor_index, direction):
|
||||
# if motor_index == 1:
|
||||
# self.motor1_direction = direction
|
||||
# elif motor_index == 2:
|
||||
# self.motor2_direction = direction
|
||||
|
||||
def parse_packet(self, base64_data):
|
||||
binary_data = base64.b64decode(base64_data)
|
||||
channel_A = []
|
||||
channel_B = []
|
||||
header_a = struct.unpack('<H', binary_data[0:2])[0]
|
||||
header_b = struct.unpack('<H', binary_data[2:4])[0]
|
||||
queue_remain = struct.unpack('<H', binary_data[4:6])[0]
|
||||
data_len = struct.unpack('<H', binary_data[6:8])[0]
|
||||
|
||||
if header_a != 0xAA55 or header_b != 0x55CC:
|
||||
print("unknown packet header",hex(header_a), hex(header_b))
|
||||
return None
|
||||
data_points = []
|
||||
offset = 8
|
||||
for _ in range(data_len):
|
||||
if offset + 4 > len(binary_data):
|
||||
print("offset:", offset, "data_len:", len(data_points))
|
||||
break
|
||||
channel_A.append(struct.unpack('<H', binary_data[offset:offset+2])[0])
|
||||
channel_B.append(struct.unpack('<H', binary_data[offset+2:offset+4])[0])
|
||||
offset += 12
|
||||
file_content = bytearray()
|
||||
|
||||
for i in range (data_len):
|
||||
file_content.extend(struct.pack('<ff', channel_A[i], channel_B[i]))
|
||||
# 解析数据单元
|
||||
return channel_A, channel_B,file_content, data_len
|
||||
|
||||
def run(self):
|
||||
"""启动 UDP 服务器"""
|
||||
self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
self.server_socket.bind((self.ip, self.port))
|
||||
|
||||
# 增加接收缓冲区大小
|
||||
self.server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 65536) # 设置为 64KB
|
||||
|
||||
print(f"UDP server is running on {self.ip}:{self.port}")
|
||||
|
||||
while self.running:
|
||||
try:
|
||||
while True: # 循环接收数据
|
||||
try:
|
||||
data, addr = self.server_socket.recvfrom(65536) # 接收最多 64KB 数据
|
||||
msg = json.loads(data.decode())
|
||||
msg_type = msg.get('type')
|
||||
if msg_type == "dev_discovery":
|
||||
self.handle_dev_discovery(msg, addr[0])
|
||||
if msg_type == "data_points":
|
||||
self.handle_data_point(msg, addr[0])
|
||||
except json.JSONDecodeError:
|
||||
print(f"Error decoding JSON data: {data.decode()}")
|
||||
except OSError as e:
|
||||
print(f"Error receiving data: {e}") # 记录错误信息
|
||||
# finally:
|
||||
# self.stop()
|
||||
|
||||
def correct_data(self, filename):
|
||||
# 加载数据
|
||||
|
||||
processor.load_data(filename)
|
||||
filename = filename[0:-4] +"_corrected.csv"
|
||||
# 自动选择圆弧区间
|
||||
if processor.auto_select_arcs():
|
||||
# 拟合数据
|
||||
if processor.fit_data():
|
||||
# 校正数据
|
||||
corrected = processor.correct_data()
|
||||
|
||||
# 保存校正后的数据,去掉头尾各200个点
|
||||
channel_A,channel_B,corrected = processor.save_corrected_data(
|
||||
corrected,
|
||||
filename,
|
||||
trim_edges=True,
|
||||
trim_amount=0
|
||||
)
|
||||
return channel_A,channel_B,corrected
|
||||
return None,None,None
|
||||
|
||||
def handle_dev_discovery(self, msg, ip_address):
|
||||
if msg.get("name") == "daq":
|
||||
status = msg.get("status")
|
||||
if status:
|
||||
# temp = status.get("temp", 0) / 10
|
||||
# speed = status.get("speed", 0)
|
||||
if ip_address == "192.168.100.236":
|
||||
self.update_acquisition_info.emit(1, status)
|
||||
elif ip_address == "192.168.100.237":
|
||||
self.update_acquisition_info.emit(2, status)
|
||||
|
||||
|
||||
def handle_data_point(self, msg, ip_address):
|
||||
try:
|
||||
print("handle data from", ip_address)
|
||||
direction = ""
|
||||
if msg.get("direction") == "up":
|
||||
direction = "UP"
|
||||
elif msg.get("direction") == "down":
|
||||
direction = "DOWN"
|
||||
if msg.get("name") == "daq":
|
||||
channelA, channelB,file_content, row_count =self.parse_packet(msg.get("data","") )
|
||||
index_col = [i for i in range(row_count)]
|
||||
#save data to csv file
|
||||
filename = f'/home/admin/PythonPro/TGX_PYSERVER/datas/data{time.strftime("%Y%m%d%H%M%S")}.csv'
|
||||
with open(filename, 'w', newline='') as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(['INDEX', 'CH1', 'CH2'])
|
||||
writer.writerows(zip(index_col, channelA, channelB))
|
||||
if COREECTMODE:
|
||||
channelA,channelB,corrected = self.correct_data(filename)
|
||||
if corrected:
|
||||
file_content = corrected
|
||||
channelA_json = json.dumps(channelA)
|
||||
channelB_json = json.dumps(channelB)
|
||||
channelA_bytes = channelA_json.encode('utf-8')
|
||||
channelB_bytes = channelB_json.encode('utf-8')
|
||||
|
||||
if ip_address == "192.168.100.236":
|
||||
self.collect_object1["method"] = "scanner1"
|
||||
self.start_cal.emit(1, file_content, direction, row_count)
|
||||
print("====emit direction[1]:", direction)
|
||||
self.collect_object1['channelA_data'] = base64.b64encode(channelA_bytes).decode('utf-8')
|
||||
self.collect_object1['channelB_data'] = base64.b64encode(channelB_bytes).decode('utf-8')
|
||||
self.send_collect_data.emit(self.collect_object1)
|
||||
elif ip_address == "192.168.100.237":
|
||||
self.collect_object2["method"] = "scanner2"
|
||||
self.start_cal.emit(2, file_content, direction, row_count)
|
||||
print("====emit direction[2]:", direction)
|
||||
self.collect_object2['channelA_data'] = base64.b64encode(channelA_bytes).decode('utf-8')
|
||||
self.collect_object2['channelB_data'] = base64.b64encode(channelB_bytes).decode('utf-8')
|
||||
self.send_collect_data.emit(self.collect_object2)
|
||||
|
||||
except OSError as e:
|
||||
print(f"Error handle_data_point: {e}") # 记录错误信息
|
||||
|
||||
def stop(self):
|
||||
"""停止 UDP 服务器"""
|
||||
self.running = False
|
||||
if self.server_socket:
|
||||
self.server_socket.close()
|
||||
self.wait() # 等待线程结束
|
||||
|
||||
def send_message(self, message, ip, port):
|
||||
"""发送消息到指定的 IP 和端口"""
|
||||
if self.server_socket:
|
||||
self.server_socket.sendto(message.encode(), (ip, port))
|
||||
|
||||
def __del__(self):
|
||||
self.stop() # 确保在对象销毁时停止服务器
|
||||
@@ -0,0 +1,485 @@
|
||||
import socket
|
||||
import base64
|
||||
import json
|
||||
import time
|
||||
import psutil
|
||||
import struct
|
||||
import csv
|
||||
import threading
|
||||
from collections import defaultdict
|
||||
from typing import Dict, List, Tuple, Optional
|
||||
import logging
|
||||
from config import *
|
||||
from x_ray_data_processor import processor
|
||||
from PySide6.QtCore import QObject, Signal, Slot, QThread, QTimer
|
||||
|
||||
# 配置日志
|
||||
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# 定义adcDataUnitDef结构体
|
||||
class adcDataUnitDef:
|
||||
def __init__(self):
|
||||
self.ucCmd = 0
|
||||
self.ucAddr = 0
|
||||
self.usData = 0
|
||||
|
||||
# 新增:UDP包头结构定义 (与C端保持一致)
|
||||
class UDPPacketHeader:
|
||||
"""UDP包头结构"""
|
||||
MAGIC = 0xAA55
|
||||
HEADER_SIZE = 16 # 包头大小
|
||||
|
||||
def __init__(self, data: bytes):
|
||||
"""
|
||||
从字节数据解析包头
|
||||
"""
|
||||
if len(data) < self.HEADER_SIZE:
|
||||
raise ValueError("数据长度不足,无法解析包头")
|
||||
|
||||
# 解析包头字段 (小端序)
|
||||
(self.magic, self.total_size, self.big_packet_id,
|
||||
self.packet_id, self.total_packets, self.payload_len) = struct.unpack('<HHIHHH', data[:16])
|
||||
|
||||
# 验证魔数
|
||||
if self.magic != self.MAGIC:
|
||||
raise ValueError(f"无效的魔数: {self.magic:04X}, 期望: {self.MAGIC:04X}")
|
||||
|
||||
# 新增:包重组器,处理单个大包的重组
|
||||
class PacketReassembler:
|
||||
"""包重组器,处理单个大包的重组"""
|
||||
|
||||
def __init__(self, big_packet_id: int, total_size: int, total_packets: int,
|
||||
sender_addr: Tuple[str, int], timeout: float = 1.0):
|
||||
self.big_packet_id = big_packet_id
|
||||
self.total_size = total_size
|
||||
self.total_packets = total_packets
|
||||
self.sender_addr = sender_addr
|
||||
self.timeout = timeout
|
||||
self.start_time = time.time()
|
||||
|
||||
# 存储接收到的包
|
||||
self.packets: Dict[int, bytes] = {}
|
||||
self.received_count = 0
|
||||
self.completed = False
|
||||
self.data = None
|
||||
|
||||
def add_packet(self, packet_id: int, payload: bytes) -> bool:
|
||||
"""
|
||||
添加一个包到重组器
|
||||
返回True如果包是新接收的,False如果包已存在
|
||||
"""
|
||||
if packet_id in self.packets:
|
||||
return False # 包已存在
|
||||
|
||||
self.packets[packet_id] = payload
|
||||
self.received_count += 1
|
||||
|
||||
# 检查是否所有包都已接收
|
||||
if self.received_count == self.total_packets:
|
||||
self._reassemble()
|
||||
return True
|
||||
|
||||
return True
|
||||
|
||||
def _reassemble(self):
|
||||
"""重组完整数据"""
|
||||
try:
|
||||
# 按包序号排序
|
||||
sorted_packets = sorted(self.packets.items())
|
||||
|
||||
# 重组数据
|
||||
reassembled_data = b''
|
||||
for packet_id, payload in sorted_packets:
|
||||
reassembled_data += payload
|
||||
|
||||
# 截取到指定长度
|
||||
self.data = reassembled_data[:self.total_size]
|
||||
self.completed = True
|
||||
|
||||
logger.info(f"大包 {self.big_packet_id} 重组完成,大小: {len(self.data)} 字节")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"重组大包 {self.big_packet_id} 时出错: {e}")
|
||||
|
||||
def is_timeout(self) -> bool:
|
||||
"""检查是否超时"""
|
||||
return time.time() - self.start_time > self.timeout
|
||||
|
||||
def is_completed(self) -> bool:
|
||||
"""检查是否完成重组"""
|
||||
return self.completed
|
||||
|
||||
def get_data(self) -> Optional[bytes]:
|
||||
"""获取重组后的数据"""
|
||||
return self.data if self.completed else None
|
||||
|
||||
class UdpServerEnhanced(QThread):
|
||||
start_cal = Signal(int, bytearray, str, int) # 信号:接收到消息,包含数据、发送者地址和端口
|
||||
send_collect_data = Signal(dict) #
|
||||
update_acquisition_info = Signal(int, dict)
|
||||
|
||||
def __init__(self, ip="0.0.0.0", port=50009, parent=None):
|
||||
super(UdpServerEnhanced, self).__init__(parent)
|
||||
self.kill_udp_port(port)
|
||||
self.ip = ip
|
||||
self.port = port
|
||||
self.server_socket = None
|
||||
self.running = True
|
||||
self.motor1_direction = ""
|
||||
self.motor2_direction = ""
|
||||
|
||||
# 新增:分包重组相关属性
|
||||
self.reassemblers: Dict[Tuple[Tuple[str, int], int], PacketReassembler] = {}
|
||||
self.reassembler_lock = threading.Lock()
|
||||
self.cleanup_thread = None
|
||||
self.timeout = 1.0 # 1秒超时
|
||||
|
||||
self.collect_object1 = {
|
||||
"method" : "scanner1",
|
||||
"channelA_data" :"base64",
|
||||
"channelB_data" :"base64",
|
||||
"intensityA" : 98,
|
||||
"intensityB" : 98
|
||||
}
|
||||
self.collect_object2 = {
|
||||
"method" : "scanner2",
|
||||
"channelA_data" :"base64",
|
||||
"channelB_data" :"base64",
|
||||
"intensityA" : 98,
|
||||
"intensityB" : 98
|
||||
}
|
||||
self.debug_timer = QTimer(self)
|
||||
self.debug_timer.timeout.connect(self.update_debug_timer)
|
||||
if YSMODE:
|
||||
self.debug_timer.start(1000)
|
||||
|
||||
def update_debug_timer(self):
|
||||
import random
|
||||
channelA = [random.randint(1, 1000) for _ in range(3000)]
|
||||
channelB = [random.randint(1, 1000) for _ in range(3000)]
|
||||
self.collect_object1['channelA_data'] = base64.b64encode(json.dumps(channelA).encode()).decode()
|
||||
self.collect_object1['channelB_data'] = base64.b64encode(json.dumps(channelB).encode()).decode()
|
||||
self.collect_object2['channelA_data'] = base64.b64encode(json.dumps(channelA).encode()).decode()
|
||||
self.collect_object2['channelB_data'] = base64.b64encode(json.dumps(channelB).encode()).decode()
|
||||
self.send_collect_data.emit(self.collect_object1)
|
||||
self.send_collect_data.emit(self.collect_object2)
|
||||
|
||||
def kill_udp_port(self, port):
|
||||
for proc in psutil.process_iter():
|
||||
try:
|
||||
for conn in proc.connections(kind='udp'):
|
||||
if conn.laddr.port == port:
|
||||
print(f" PID={proc.pid}, procname={proc.name()}")
|
||||
proc.kill()
|
||||
except psutil.AccessDenied:
|
||||
continue
|
||||
|
||||
def packet_crc(self, data):
|
||||
usSum = 0
|
||||
for byte in data:
|
||||
usSum += byte
|
||||
return usSum & 0xFFFF # 确保返回值为16位
|
||||
|
||||
def handle_sample_data(self, data):
|
||||
# 确保数据长度是4的倍数
|
||||
if len(data) % 4 != 0:
|
||||
raise ValueError("Data length must be a multiple of 4")
|
||||
|
||||
# 使用列表推导式来简化代码
|
||||
unpacked_data = [struct.unpack_from('<H', data[i:i+2])[0] for i in range(0, len(data), 2)]
|
||||
|
||||
# 将数据分为两组
|
||||
data_list1 = unpacked_data[::2] # 取偶数索引的元素
|
||||
data_list2 = unpacked_data[1::2] # 取奇数索引的元素
|
||||
|
||||
# 打印数据长度,用于调试
|
||||
print(f"Data contains {len(unpacked_data) // 2} pairs of values")
|
||||
# print(data_list1)
|
||||
# print("=======================================")
|
||||
# print(data_list2)
|
||||
file_content = bytearray()
|
||||
datalen = len(data_list2)
|
||||
for i in range (datalen):
|
||||
file_content.extend(struct.pack('<ff', data_list1[i], data_list2[i]))
|
||||
return data_list1,data_list2,file_content,datalen
|
||||
|
||||
def parse_packet(self, base64_data):
|
||||
binary_data = base64.b64decode(base64_data)
|
||||
channel_A = []
|
||||
channel_B = []
|
||||
header_a = struct.unpack('<H', binary_data[0:2])[0]
|
||||
header_b = struct.unpack('<H', binary_data[2:4])[0]
|
||||
queue_remain = struct.unpack('<H', binary_data[4:6])[0]
|
||||
data_len = struct.unpack('<H', binary_data[6:8])[0]
|
||||
|
||||
if header_a != 0xAA55 or header_b != 0x55CC:
|
||||
print("unknown packet header",hex(header_a), hex(header_b))
|
||||
return None
|
||||
data_points = []
|
||||
offset = 8
|
||||
for _ in range(data_len):
|
||||
if offset + 4 > len(binary_data):
|
||||
print("offset:", offset, "data_len:", len(data_points))
|
||||
break
|
||||
channel_A.append(struct.unpack('<H', binary_data[offset:offset+2])[0])
|
||||
channel_B.append(struct.unpack('<H', binary_data[offset+2:offset+4])[0])
|
||||
offset += 12
|
||||
file_content = bytearray()
|
||||
|
||||
for i in range (data_len):
|
||||
file_content.extend(struct.pack('<ff', channel_A[i], channel_B[i]))
|
||||
# 解析数据单元
|
||||
return channel_A, channel_B,file_content, data_len
|
||||
|
||||
# 新增:检查是否为分包数据
|
||||
def is_fragmented_packet(self, data: bytes) -> bool:
|
||||
"""检查数据包是否以UDP_PACKET_MAGIC开头"""
|
||||
if len(data) < 2:
|
||||
return False
|
||||
magic = struct.unpack('<H', data[:2])[0]
|
||||
return magic == UDPPacketHeader.MAGIC
|
||||
|
||||
# 新增:处理分包数据
|
||||
def handle_fragmented_packet(self, data: bytes, addr: Tuple[str, int]):
|
||||
"""处理分包数据"""
|
||||
try:
|
||||
# 解析包头
|
||||
header = UDPPacketHeader(data)
|
||||
|
||||
# 提取载荷数据
|
||||
payload = data[UDPPacketHeader.HEADER_SIZE:UDPPacketHeader.HEADER_SIZE + header.payload_len]
|
||||
|
||||
# 获取或创建重组器
|
||||
reassembler = self._get_or_create_reassembler(header, addr)
|
||||
|
||||
if reassembler is None:
|
||||
return # 超时或无效的包
|
||||
|
||||
# 添加包到重组器
|
||||
if reassembler.add_packet(header.packet_id, payload):
|
||||
logger.debug(f"接收包 {header.packet_id}/{header.total_packets} "
|
||||
f"来自 {addr}, 大包ID: {header.big_packet_id}")
|
||||
|
||||
# 检查是否完成重组
|
||||
if reassembler.is_completed():
|
||||
self._handle_completed_packet(reassembler)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"处理来自 {addr} 的分包数据时出错: {e}")
|
||||
|
||||
# 新增:获取或创建包重组器
|
||||
def _get_or_create_reassembler(self, header: UDPPacketHeader, addr: Tuple[str, int]) -> Optional[PacketReassembler]:
|
||||
"""获取或创建包重组器"""
|
||||
key = (addr, header.big_packet_id)
|
||||
|
||||
with self.reassembler_lock:
|
||||
# 检查是否已存在
|
||||
if key in self.reassemblers:
|
||||
reassembler = self.reassemblers[key]
|
||||
|
||||
# 检查是否超时
|
||||
if reassembler.is_timeout():
|
||||
logger.warning(f"大包 {header.big_packet_id} 超时,丢弃")
|
||||
del self.reassemblers[key]
|
||||
return None
|
||||
|
||||
return reassembler
|
||||
|
||||
# 创建新的重组器
|
||||
reassembler = PacketReassembler(
|
||||
header.big_packet_id,
|
||||
header.total_size,
|
||||
header.total_packets,
|
||||
addr,
|
||||
self.timeout
|
||||
)
|
||||
|
||||
self.reassemblers[key] = reassembler
|
||||
logger.info(f"创建新的大包重组器: ID={header.big_packet_id}, "
|
||||
f"大小={header.total_size}, 包数={header.total_packets}, 来源={addr}")
|
||||
|
||||
return reassembler
|
||||
|
||||
# 新增:处理完成重组的数据包
|
||||
def _handle_completed_packet(self, reassembler: PacketReassembler):
|
||||
"""处理完成重组的数据包"""
|
||||
data = reassembler.get_data()
|
||||
if data:
|
||||
logger.info(f"收到完整数据包: 大小={len(data)} 字节, "
|
||||
f"来源={reassembler.sender_addr}, ID={reassembler.big_packet_id}")
|
||||
|
||||
# 调用现有的数据处理方法
|
||||
try:
|
||||
# 尝试解析为JSON格式
|
||||
msg = json.loads(data.decode().strip('\0'))
|
||||
msg_type = msg.get('type')
|
||||
if msg_type == "dev_discovery":
|
||||
self.handle_dev_discovery(msg, reassembler.sender_addr[0])
|
||||
elif msg_type == "data_points":
|
||||
self.handle_data_point(msg, reassembler.sender_addr[0])
|
||||
else:
|
||||
logger.warning(f"未知的消息类型: {msg_type}")
|
||||
except json.JSONDecodeError:
|
||||
logger.warning(f"重组后的数据不是有效的JSON格式")
|
||||
except Exception as e:
|
||||
logger.error(f"处理重组数据时出错: {e}")
|
||||
|
||||
# 从重组器列表中移除
|
||||
with self.reassembler_lock:
|
||||
key = (reassembler.sender_addr, reassembler.big_packet_id)
|
||||
if key in self.reassemblers:
|
||||
del self.reassemblers[key]
|
||||
|
||||
# 新增:清理超时的包
|
||||
def _cleanup_timeout_packets(self):
|
||||
"""清理超时的包(在后台线程中运行)"""
|
||||
while self.running:
|
||||
time.sleep(0.5) # 每0.5秒检查一次
|
||||
|
||||
with self.reassembler_lock:
|
||||
timeout_keys = []
|
||||
for key, reassembler in self.reassemblers.items():
|
||||
if reassembler.is_timeout():
|
||||
timeout_keys.append(key)
|
||||
|
||||
# 移除超时的包
|
||||
for key in timeout_keys:
|
||||
reassembler = self.reassemblers[key]
|
||||
logger.warning(f"清理超时的大包: ID={reassembler.big_packet_id}, "
|
||||
f"来源={reassembler.sender_addr}, "
|
||||
f"已接收={reassembler.received_count}/{reassembler.total_packets}")
|
||||
del self.reassemblers[key]
|
||||
|
||||
def run(self):
|
||||
"""启动 UDP 服务器"""
|
||||
self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
self.server_socket.bind((self.ip, self.port))
|
||||
|
||||
# 增加接收缓冲区大小
|
||||
self.server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 65536) # 设置为 64KB
|
||||
|
||||
print(f"UDP server is running on {self.ip}:{self.port}")
|
||||
|
||||
# 启动清理线程
|
||||
self.cleanup_thread = threading.Thread(target=self._cleanup_timeout_packets, daemon=True)
|
||||
self.cleanup_thread.start()
|
||||
|
||||
while self.running:
|
||||
try:
|
||||
while True: # 循环接收数据
|
||||
try:
|
||||
data, addr = self.server_socket.recvfrom(65536) # 接收最多 64KB 数据
|
||||
|
||||
# 检查是否为分包数据
|
||||
if self.is_fragmented_packet(data):
|
||||
# 处理分包数据
|
||||
self.handle_fragmented_packet(data, addr)
|
||||
else:
|
||||
# 使用现有的处理方法
|
||||
try:
|
||||
msg = json.loads(data.decode().strip('\0'))
|
||||
msg_type = msg.get('type')
|
||||
if msg_type == "dev_discovery":
|
||||
self.handle_dev_discovery(msg, addr[0])
|
||||
if msg_type == "data_points":
|
||||
self.handle_data_point(msg, addr[0])
|
||||
except json.JSONDecodeError:
|
||||
print(f"Error decoding JSON data: {data.decode()}")
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error processing data: {e}")
|
||||
|
||||
except OSError as e:
|
||||
print(f"Error receiving data: {e}") # 记录错误信息
|
||||
|
||||
def correct_data(self, filename):
|
||||
# 加载数据
|
||||
processor.load_data(filename)
|
||||
filename = filename[0:-4] +"_corrected.csv"
|
||||
# 自动选择圆弧区间
|
||||
if processor.auto_select_arcs():
|
||||
# 拟合数据
|
||||
if processor.fit_data():
|
||||
# 校正数据
|
||||
corrected = processor.correct_data()
|
||||
|
||||
# 保存校正后的数据,去掉头尾各200个点
|
||||
channel_A,channel_B,corrected = processor.save_corrected_data(
|
||||
corrected,
|
||||
filename,
|
||||
trim_edges=True,
|
||||
trim_amount=0
|
||||
)
|
||||
return channel_A,channel_B,corrected
|
||||
return None,None,None
|
||||
|
||||
def handle_dev_discovery(self, msg, ip_address):
|
||||
if msg.get("name") == "daq":
|
||||
status = msg.get("status")
|
||||
if status:
|
||||
# temp = status.get("temp", 0) / 10
|
||||
# speed = status.get("speed", 0)
|
||||
if ip_address == "192.168.100.236":
|
||||
self.update_acquisition_info.emit(1, status)
|
||||
elif ip_address == "192.168.100.237":
|
||||
self.update_acquisition_info.emit(2, status)
|
||||
|
||||
def handle_data_point(self, msg, ip_address):
|
||||
try:
|
||||
print("handle data from", ip_address)
|
||||
direction = ""
|
||||
if msg.get("direction") == "up":
|
||||
direction = "UP"
|
||||
elif msg.get("direction") == "down":
|
||||
direction = "DOWN"
|
||||
if msg.get("name") == "daq":
|
||||
channelA, channelB,file_content, row_count =self.parse_packet(msg.get("data","") )
|
||||
index_col = [i for i in range(row_count)]
|
||||
#save data to csv file
|
||||
filename = f'/home/admin/PythonPro/TGX_PYSERVER/datas/data{time.strftime("%Y%m%d%H%M%S")}.csv'
|
||||
with open(filename, 'w', newline='') as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(['INDEX', 'CH1', 'CH2'])
|
||||
writer.writerows(zip(index_col, channelA, channelB))
|
||||
if COREECTMODE:
|
||||
channelA,channelB,corrected = self.correct_data(filename)
|
||||
if corrected:
|
||||
file_content = corrected
|
||||
channelA_json = json.dumps(channelA)
|
||||
channelB_json = json.dumps(channelB)
|
||||
channelA_bytes = channelA_json.encode('utf-8')
|
||||
channelB_bytes = channelB_json.encode('utf-8')
|
||||
|
||||
if ip_address == "192.168.100.236":
|
||||
self.collect_object1["method"] = "scanner1"
|
||||
self.start_cal.emit(1, file_content, direction, row_count)
|
||||
print("====emit direction[1]:", direction)
|
||||
self.collect_object1['channelA_data'] = base64.b64encode(channelA_bytes).decode('utf-8')
|
||||
self.collect_object1['channelB_data'] = base64.b64encode(channelB_bytes).decode('utf-8')
|
||||
self.send_collect_data.emit(self.collect_object1)
|
||||
elif ip_address == "192.168.100.237":
|
||||
self.collect_object2["method"] = "scanner2"
|
||||
self.start_cal.emit(2, file_content, direction, row_count)
|
||||
print("====emit direction[2]:", direction)
|
||||
self.collect_object2['channelA_data'] = base64.b64encode(channelA_bytes).decode('utf-8')
|
||||
self.collect_object2['channelB_data'] = base64.b64encode(channelB_bytes).decode('utf-8')
|
||||
self.send_collect_data.emit(self.collect_object2)
|
||||
|
||||
except OSError as e:
|
||||
print(f"Error handle_data_point: {e}") # 记录错误信息
|
||||
|
||||
def stop(self):
|
||||
"""停止 UDP 服务器"""
|
||||
self.running = False
|
||||
if self.server_socket:
|
||||
self.server_socket.close()
|
||||
self.wait() # 等待线程结束
|
||||
|
||||
def send_message(self, message, ip, port):
|
||||
"""发送消息到指定的 IP 和端口"""
|
||||
if self.server_socket:
|
||||
self.server_socket.sendto(message.encode(), (ip, port))
|
||||
|
||||
def __del__(self):
|
||||
self.stop() # 确保在对象销毁时停止服务器
|
||||
@@ -0,0 +1,78 @@
|
||||
#include "main.h"
|
||||
#include "cmsis_os.h"
|
||||
#include "lwip.h"
|
||||
|
||||
/* Private includes ----------------------------------------------------------*/
|
||||
/* USER CODE BEGIN Includes */
|
||||
#include "string.h"
|
||||
#include "stm32h7xx_hal.h"
|
||||
#include "stm32h7xx_it.h"
|
||||
#include <stdlib.h>
|
||||
#include <time.h>
|
||||
|
||||
#include <lwip/sockets.h>
|
||||
#include "lwip/sys.h"
|
||||
#include "lwip/api.h"
|
||||
|
||||
#include "IAP/iap.h"
|
||||
#include "udpp.h"
|
||||
|
||||
|
||||
int udpSendTo(uint8_t *buf, int len, char *addr, uint16_t port);
|
||||
|
||||
|
||||
int udpSendBigPacket(uint8_t *buf, int len, char *addr, uint16_t port)
|
||||
{
|
||||
// 参数检查
|
||||
if (buf == NULL || len <= 0 || len > UDP_MAX_TOTAL_SIZE || addr == NULL) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
// 生成随机大包ID
|
||||
uint32_t big_packet_id = (uint32_t)rand();
|
||||
|
||||
// 计算需要的包数量
|
||||
int total_packets = (len + UDP_MAX_PAYLOAD_SIZE - 1) / UDP_MAX_PAYLOAD_SIZE;
|
||||
|
||||
// 分配发送缓冲区
|
||||
static uint8_t send_buf[UDP_MAX_PACKET_SIZE];
|
||||
|
||||
int result = 0;
|
||||
int offset = 0;
|
||||
|
||||
// 逐个发送包
|
||||
for (int packet_id = 0; packet_id < total_packets; packet_id++) {
|
||||
// 准备包头
|
||||
udp_packet_header_t *header = (udp_packet_header_t *)send_buf;
|
||||
header->magic = UDP_PACKET_MAGIC;
|
||||
header->total_size = len;
|
||||
header->big_packet_id = big_packet_id;
|
||||
header->packet_id = packet_id;
|
||||
header->total_packets = total_packets;
|
||||
|
||||
// 计算当前包的有效载荷长度
|
||||
int remaining_data = len - offset;
|
||||
int payload_len = (remaining_data > UDP_MAX_PAYLOAD_SIZE) ? UDP_MAX_PAYLOAD_SIZE : remaining_data;
|
||||
header->payload_len = payload_len;
|
||||
|
||||
// 填充保留字段为0
|
||||
memset(header->reserved, 0, sizeof(header->reserved));
|
||||
|
||||
// 复制数据到发送缓冲区
|
||||
memcpy(send_buf + UDP_HEADER_SIZE, buf + offset, payload_len);
|
||||
|
||||
// 发送包
|
||||
int send_len = UDP_HEADER_SIZE + payload_len;
|
||||
int send_result = udpSendTo(send_buf, send_len, addr, port);
|
||||
|
||||
if (send_result < 0) {
|
||||
result = send_result;
|
||||
break;
|
||||
}
|
||||
|
||||
offset += payload_len;
|
||||
osDelay(5);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
#ifndef UDP_H
|
||||
#define UDP_H
|
||||
|
||||
/* Includes ------------------------------------------------------------------*/
|
||||
#include <stdint.h>
|
||||
#include <stdbool.h>
|
||||
|
||||
/* Exported types ------------------------------------------------------------*/
|
||||
|
||||
// 包头结构定义
|
||||
typedef struct {
|
||||
uint16_t magic; // 魔数,用于标识包的有效性 (0xAA55)
|
||||
uint16_t total_size; // 总数据大小 (字节)
|
||||
uint32_t big_packet_id; // 大包ID,32位随机整数,用于区分不同的大包
|
||||
uint16_t packet_id; // 包序号 (0, 1, 2, ...)
|
||||
uint16_t total_packets; // 总包数
|
||||
uint16_t payload_len; // 当前包的有效载荷长度
|
||||
uint8_t reserved[2]; // 保留字段,用于未来扩展
|
||||
} __attribute__((packed)) udp_packet_header_t;
|
||||
|
||||
// 函数声明
|
||||
int udpSendTo(uint8_t *buf, int len, char *addr, uint16_t port);
|
||||
int udpSendBigPacket(uint8_t *buf, int len, char *addr, uint16_t port);
|
||||
|
||||
// 常量定义
|
||||
#define UDP_PACKET_MAGIC 0xAA55
|
||||
#define UDP_MAX_PAYLOAD_SIZE 512
|
||||
#define UDP_HEADER_SIZE sizeof(udp_packet_header_t)
|
||||
#define UDP_MAX_PACKET_SIZE (UDP_HEADER_SIZE + UDP_MAX_PAYLOAD_SIZE)
|
||||
#define UDP_MAX_TOTAL_SIZE 60*1024 // 60KB
|
||||
|
||||
#endif // UDP_H
|
||||
Reference in New Issue
Block a user