一月二日 等待变化等待机会
[Interface]
# 服务器私钥
PrivateKey = ...
# 服务器在Wireguard网络中的IP地址
Address = 10.0.0.1/24
ListenPort = 5182
PostUp= iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
# 定义客户端
[Peer]
# 客户端公钥
PublicKey =...
# 允许客户端访问的IP地址范围
AllowedIPs = 10.0.0.2/32
$ ip route
default via 172.31.16.1 dev eth0 proto dhcp src 172.31.29.227 metric 100
10.0.0.0/24 dev wg0 proto kernel scope link src 10.0.0.1
172.31.0.2 via 172.31.16.1 dev eth0 proto dhcp src 172.31.29.227 metric 100
172.31.16.0/20 dev eth0 proto kernel scope link src 172.31.29.227 metric 100
172.31.16.1 dev eth0 proto dhcp scope link src 172.31.29.227 metric 100
[Interface]
# 客户端私钥
PrivateKey = ...
# 客户端在Wireguard网络中的IP地址
Address = 10.0.0.2/24
DNS=172.31.29.227
[Peer]
# 服务器公钥
PublicKey = ...
# 服务器的公网IP地
Endpoint = 54.146.251.127:5182
AllowedIPs = 0.0.0.0/0
$ resolvectl
Global
Protocols: -LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported
resolv.conf mode: foreign
Current DNS Server: 10.0.0.1
DNS Servers: 10.0.0.1
Link 2 (enp0s31f6)
Current Scopes: DNS
Protocols: +DefaultRoute +LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported
Current DNS Server: 218.85.157.99
DNS Servers: 218.85.152.99 218.85.157.99 fe80::1%23667
Link 3 (wlp109s0)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported
Link 766 (wg0)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS DNSOverTLS=opportunistic DNSSEC=no/unsupported
$ ip route
0.0.0.0/1 via 10.0.0.1 dev wg0
default via 192.168.1.1 dev enp0s31f6 proto dhcp metric 100
10.0.0.0/24 dev wg0 proto kernel scope link src 10.0.0.2
54.146.251.127 via 192.168.1.1 dev enp0s31f6
169.254.0.0/16 dev enp0s31f6 scope link metric 1000
192.168.1.0/24 dev enp0s31f6 proto kernel scope link src 192.168.1.6 metric 100
sudo ip route add 0.0.0.0/1 dev wg0
sudo ip route add 128.0.0.0/1 dev wg0
Is wireguard server "Forwarding" mainly for firewall?
That's a very good question that gets at the heart of how networking works.
While it's true that forwarding is a core function of a firewall, it's not exclusively a firewall function. In the context of a WireGuard server (or any router), "forwarding" refers to the basic network process of taking a packet received on one network interface and sending it out another network interface.
Here's a breakdown:
Forwarding in General:
- Core Routing Function: At its most fundamental level, forwarding is what routers (including your WireGuard server) do. When a packet arrives at a router, it examines the destination IP address.3 If the destination is on a different network than the incoming interface, the router "forwards" the packet to the appropriate outgoing interface.
- Essential for Inter-Network Communication: Without forwarding, devices on different networks (like your home network and the internet) could not communicate with each other.
Forwarding in the Context of a WireGuard Server:
- Connecting Private and Public Networks: Your WireGuard server acts as a bridge between the private WireGuard network (e.g., 10.8.0.0/24) and the public internet.
- iptables -A FORWARD -i wg0 -j ACCEPT: This specific iptables rule tells the Linux kernel (which is the operating system of your server) to allow packets to be forwarded from the WireGuard interface (wg0) to other interfaces (and vice versa). This is absolutely essential for your WireGuard clients to be able to reach the internet.
Relationship to Firewalls:
- Firewalls Use Forwarding: Firewalls use forwarding as a mechanism to inspect and filter traffic.5 A firewall can have rules that say "forward packets from this IP to that IP only if they are on port 80 (HTTP)" or "block all forwarding from this network to that network."
- Forwarding Exists Independently of Firewalls: The basic act of forwarding is a lower-level network function that exists regardless of whether you have a complex firewall setup. Even with no explicit firewall rules, the kernel still needs to know whether to forward packets or not.
Analogy
:Think of a post office:
- Forwarding: The basic act of sorting mail and sending it to the correct destination is like forwarding.
- Firewall: A security guard at the post office who checks packages for suspicious contents and decides whether to allow them to be delivered is like a firewall.
The post office needs to forward mail even without a security guard. The security guard (firewall) uses the forwarding system to do its job of filtering.
In summary:
Therefore, even if you don't have a complex firewall configuration, you still need to enable forwarding on your WireGuard server for it to function correctly. The iptables -A FORWARD ... rule is essential for this.
- Forwarding is a fundamental network function that allows devices on different networks to communicate.
- WireGuard servers rely on forwarding to connect clients on the private VPN network to the public internet.
- Firewalls use forwarding as a mechanism to filter traffic, but forwarding itself is not solely a firewall function.
listen-on { 10.0.0.1; }; // Listen on the WireGuard server's internal IP
allow-query { 10.0.0.0/24; }; // Allow queries from the WireGuard network
forwarders { 8.8.8.8; 1.1.1.1; }; // Forward unresolved queries (Google and Cloudflare DNS)
//listen-on-v6 { any; }; #Comment this line to disable ipv6 listening
一月二十三日 等待变化等待机会
浴景遐思 翠树华亭掩香汤， 霓裳才解玉肌凉。 纤纤葱笋扬春水， 袅袅胴体隔水香。 暖雾氤氲思旧事， 柔波潋滟梦情长。 待得浴罢添新韵， 浅笑嫣然映景光。
ffmpeg -f alsa -i hw:1 /tmp/output.mp3
一月二十六日 等待变化等待机会
https://dl.winehq.org/wine-builds/Release.key这个旧的做法被所谓的
keyring取代了。
$ sudo wget -o /etc/apt/keyrings/winehq-archive.key https://dl.winehq.org/wine-builds/winehq.key
$ wget -np /etc/apt/sources.list.d/ https://dl.winehq.org/wine-builds/ubuntu/dists/jammy/winehq-jammy.sources
$ $ ffmpeg -i deepseek-2025-01-26_17.20.48.mp4 -vf "setpts=0.5*PTS" wine-fix.mp4
二月二日 等待变化等待机会
$ ./llama-tokenize -m /usr/share/ollama/.ollama/models/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 --prompt "how are you" 2>&1 | grep general.name
general.name str = DeepSeek R1 Distill Qwen 32B
print_info: general.name = DeepSeek R1 Distill Qwen 32B
二月十五日 等待变化等待机会
二月十九日 等待变化等待机会
On Ubuntu, Ollama saves models by default to /usr/share/ollama/.ollama/models.之前我尝试使用软链接，可是不行。这个内容是重要的，我之前没有修订用户就不可以sudo mkdir -p /etc/systemd/system/ollama.service.d; sudo nano /etc/systemd/system/ollama.service.d/override.conf[Service] User=nick Group=nick Environment="OLLAMA_MODELS=/home/nick/workspace/OLLAMA/.ollama/models"
sudo swapoff -a
sudo dd if=/dev/zero of=/swapfile bs=1M count=8192
二月二十四日 等待变化等待机会
三月四日 等待变化等待机会
$ sudo vgdisplay
--- Volume group ---
VG Name ubuntu-vg
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size <1.82 TiB
PE Size 4.00 MiB
Total PE 476018
Alloc PE / Size 476018 / <1.82 TiB
Free PE / Size 0 / 0
VG UUID fvQTI5-0C8Q-9eVF-VkQ0-fmgh-w5XH-GewwFP
$ sudo lvdisplay
--- Logical volume ---
LV Path /dev/ubuntu-vg/ubuntu-lv
LV Name ubuntu-lv
VG Name ubuntu-vg
LV UUID 3LexV3-Imjh-NB0K-oGf0-gVbz-PiF8-3S8Non
LV Write Access read/write
LV Creation host, time ubuntu-server, 2025-02-22 07:55:25 +0000
LV Status available
# open 1
LV Size 100.00 GiB
Current LE 25600
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
$ sudo pvcreate /dev/sda3
$ sudo vgextend ubuntu-vg /dev/sda3
$ sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
$ sudo resize2fs /dev/ubuntu-vg/ubuntu-lv
$ df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 101G 1.9M 101G 1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 1.8T 12G 1.8T 1% /
tmpfs 504G 0 504G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda2 2.0G 246M 1.6G 14% /boot
/dev/sda1 1.1G 6.1M 1.1G 1% /boot/efi
tmpfs 101G 4.0K 101G 1% /run/user/1000
三月五日 等待变化等待机会
sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo usermod -a -G ollama $(whoami)
$ cat /etc/passwd | grep ollama
ollama:x:998:998::/usr/share/ollama:/bin/false
$ sudo ls -dal /usr/share/ollama/
drwxr-x--- 2 ollama ollama 4096 Mar 4 22:11 /usr/share/ollama/
[Unit]
Description=Ollama Service
After=network-online.target
[Service]
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"
[Install]
WantedBy=default.target
sudo systemctl daemon-reload
sudo systemctl enable ollama
三月十一日 等待变化等待机会
build: 4858 (1e2f78a0) with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_loader: loaded meta data with 42 key-value pairs and 1025 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-9801e7fce27dbf3d0bfb468b7b21f1d132131a546dfc43e50518631b8b1800a9 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = deepseek2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.size_label str = 256x20B
llama_model_loader: - kv 3: deepseek2.block_count u32 = 61
llama_model_loader: - kv 4: deepseek2.context_length u32 = 163840
llama_model_loader: - kv 5: deepseek2.embedding_length u32 = 7168
llama_model_loader: - kv 6: deepseek2.feed_forward_length u32 = 18432
llama_model_loader: - kv 7: deepseek2.attention.head_count u32 = 128
llama_model_loader: - kv 8: deepseek2.attention.head_count_kv u32 = 128
llama_model_loader: - kv 9: deepseek2.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 10: deepseek2.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 11: deepseek2.expert_used_count u32 = 8
llama_model_loader: - kv 12: deepseek2.leading_dense_block_count u32 = 3
llama_model_loader: - kv 13: deepseek2.vocab_size u32 = 129280
llama_model_loader: - kv 14: deepseek2.attention.q_lora_rank u32 = 1536
llama_model_loader: - kv 15: deepseek2.attention.kv_lora_rank u32 = 512
llama_model_loader: - kv 16: deepseek2.attention.key_length u32 = 192
llama_model_loader: - kv 17: deepseek2.attention.value_length u32 = 128
llama_model_loader: - kv 18: deepseek2.expert_feed_forward_length u32 = 2048
llama_model_loader: - kv 19: deepseek2.expert_count u32 = 256
llama_model_loader: - kv 20: deepseek2.expert_shared_count u32 = 1
llama_model_loader: - kv 21: deepseek2.expert_weights_scale f32 = 2.500000
llama_model_loader: - kv 22: deepseek2.expert_weights_norm bool = true
llama_model_loader: - kv 23: deepseek2.expert_gating_func u32 = 2
llama_model_loader: - kv 24: deepseek2.rope.dimension_count u32 = 64
llama_model_loader: - kv 25: deepseek2.rope.scaling.type str = yarn
llama_model_loader: - kv 26: deepseek2.rope.scaling.factor f32 = 40.000000
llama_model_loader: - kv 27: deepseek2.rope.scaling.original_context_length u32 = 4096
llama_model_loader: - kv 28: deepseek2.rope.scaling.yarn_log_multiplier f32 = 0.100000
llama_model_loader: - kv 29: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 30: tokenizer.ggml.pre str = deepseek-v3
llama_model_loader: - kv 31: tokenizer.ggml.tokens arr[str,129280] = ["<｜begin▁of▁sentence｜>", "<�...
llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,129280] = [3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,127741] = ["Ġ t", "Ġ a", "i n", "Ġ Ġ", "h e...
llama_model_loader: - kv 34: tokenizer.ggml.bos_token_id u32 = 0
llama_model_loader: - kv 35: tokenizer.ggml.eos_token_id u32 = 1
llama_model_loader: - kv 36: tokenizer.ggml.padding_token_id u32 = 1
llama_model_loader: - kv 37: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 38: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 39: tokenizer.chat_template str = {% if not add_generation_prompt is de...
llama_model_loader: - kv 40: general.quantization_version u32 = 2
llama_model_loader: - kv 41: general.file_type u32 = 15
llama_model_loader: - type f32: 361 tensors
llama_model_loader: - type q4_K: 606 tensors
llama_model_loader: - type q6_K: 58 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_K - Medium
print_info: file size = 376.65 GiB (4.82 BPW)
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
load: special tokens cache size = 818
load: token to piece cache size = 0.8223 MB
print_info: arch = deepseek2
print_info: vocab_only = 0
print_info: n_ctx_train = 163840
print_info: n_embd = 7168
print_info: n_layer = 61
print_info: n_head = 128
print_info: n_head_kv = 128
print_info: n_rot = 64
print_info: n_swa = 0
print_info: n_embd_head_k = 192
print_info: n_embd_head_v = 128
print_info: n_gqa = 1
print_info: n_embd_k_gqa = 24576
print_info: n_embd_v_gqa = 16384
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: n_ff = 18432
print_info: n_expert = 256
print_info: n_expert_used = 8
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 0
print_info: rope scaling = yarn
print_info: freq_base_train = 10000.0
print_info: freq_scale_train = 0.025
print_info: n_ctx_orig_yarn = 4096
print_info: rope_finetuned = unknown
print_info: ssm_d_conv = 0
print_info: ssm_d_inner = 0
print_info: ssm_d_state = 0
print_info: ssm_dt_rank = 0
print_info: ssm_dt_b_c_rms = 0
print_info: model type = 671B
print_info: model params = 671.03 B
print_info: general.name = n/a
print_info: n_layer_dense_lead = 3
print_info: n_lora_q = 1536
print_info: n_lora_kv = 512
print_info: n_ff_exp = 2048
print_info: n_expert_shared = 1
print_info: expert_weights_scale = 2.5
print_info: expert_weights_norm = 1
print_info: expert_gating_func = sigmoid
print_info: rope_yarn_log_mul = 0.1000
print_info: vocab type = BPE
print_info: n_vocab = 129280
print_info: n_merges = 127741
print_info: BOS token = 0 '<｜begin▁of▁sentence｜>'
print_info: EOS token = 1 '<｜end▁of▁sentence｜>'
print_info: EOT token = 1 '<｜end▁of▁sentence｜>'
print_info: PAD token = 1 '<｜end▁of▁sentence｜>'
print_info: LF token = 201 'Ċ'
print_info: FIM PRE token = 128801 '<｜fim▁begin｜>'
print_info: FIM SUF token = 128800 '<｜fim▁hole｜>'
print_info: FIM MID token = 128802 '<｜fim▁end｜>'
print_info: EOG token = 1 '<｜end▁of▁sentence｜>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
#define GGML_ABORT(...) ggml_abort(__FILE__, __LINE__, __VA_ARGS__)
#define GGML_ASSERT(x) if (!(x)) GGML_ABORT("GGML_ASSERT(%s) failed", #x)
GGML_ASSERT(tokenizer && "Tokenizer not initialized. Call llama_vocab::init_tokenizer() first.");
if (!(tokenizer && "Tokenizer not initialized. Call llama_vocab::init_tokenizer() first."))
ggml_abort("/home/nick/workspace/llama.cpp/src/llama-vocab.cpp", 2330, "GGML_ASSERT(%s) failed",
"tokenizer && \"Tokenizer not initialized. Call llama_vocab::init_tokenizer() first.\"")
void ggml_abort(const char * file, int line, const char * fmt, ...) {
fflush(stdout);
fprintf(stderr, "%s:%d: ", file, line);
va_list args;
va_start(args, fmt);
vfprintf(stderr, fmt, args);
va_end(args);
fprintf(stderr, "\n");
ggml_print_backtrace();
abort();
}
static void ggml_print_backtrace(void) {
const char * GGML_NO_BACKTRACE = getenv("GGML_NO_BACKTRACE");
if (GGML_NO_BACKTRACE) {
return;
}
char attach[32];
snprintf(attach, sizeof(attach), "attach %d", getpid());
int pid = fork();
if (pid == 0) {
// try gdb
execlp("gdb", "gdb", "--batch",
"-ex", "set style enabled on",
"-ex", attach,
"-ex", "bt -frame-info source-and-location",
"-ex", "detach",
"-ex", "quit",
(char *) NULL);
//后面是其他平台的做法
所以，临时改动一下
- 0 (classic ptrace permissions): Anyone can ptrace any process, provided they have the necessary permissions (e.g., same user ID). This is the least secure setting.
- 1 (restricted ptrace): A process can only be ptraced by a parent process or a process running as root. This is the default setting in many modern Linux distributions, including Ubuntu 22.04.
- 2 (restricted ptrace admin): Only root can ptrace processes.
- 3 (no ptrace): No process can be ptraced, even by root.
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
kernel.yama.ptrace_scope = 0
这个就是神奇的工具函数！The backtrace function in the C/C++ standard library (specifically, in the GNU C Library, glibc) works by inspecting the call stack of the current process. It does not use the ptrace system call. Here's how backtrace typically works: 1. Stack Frame Inspection: The function walks up the call stack by inspecting the frame pointer (or base pointer) registers. Each stack frame contains: The return address (the address of the instruction to execute after the function returns). Local variables and function arguments. A pointer to the previous stack frame. backtrace uses these frame pointers to trace the sequence of function calls that led to the current point in the program. 2. Symbol Resolution (Optional): The backtrace_symbols function (which often accompanies backtrace) takes the addresses obtained from backtrace and attempts to resolve them to function names, source file names, and line numbers. This symbol resolution typically involves: Reading debugging information from the executable file (e.g., DWARF information). Using symbol tables to map addresses to function names. 3. No ptrace Involvement: backtrace operates within the context of the current process. It inspects the process's own memory (the call stack). ptrace is used for debugging other processes or for controlling the execution of a child process. backtrace doesn't need to do that. Therefore, backtrace does not require any special privileges or capabilities. Key Points: backtrace is a self-inspection mechanism. It relies on the structure of the call stack, which is a fundamental part of how programs execute. Symbol resolution (using backtrace_symbols) enhances the output of backtrace by providing human-readable information. In summary: backtrace is a valuable tool for debugging and profiling, and it works by directly examining the process's call stack, without the need for the ptrace system call.
/* Store up to SIZE return address of the current program state in
ARRAY and return the exact number of values stored. */
extern int backtrace (void **__array, int __size) __nonnull ((1));
/* Return names of functions from the backtrace list in ARRAY in a newly
malloc()ed memory block. */
extern char **backtrace_symbols (void *const *__array, int __size)
__THROW __nonnull ((1));
/* The nonnull function attribute marks pointer parameters that
must not be NULL. This has the name __nonnull in glibc,
and __attribute_nonnull__ in files shared with Gnulib to avoid
collision with a different __nonnull in DragonFlyBSD 5.9. */
#ifndef __attribute_nonnull__
# if __GNUC_PREREQ (3,3) || __glibc_has_attribute (__nonnull__)
# define __attribute_nonnull__(params) __attribute__ ((__nonnull__ params))
# else
# define __attribute_nonnull__(params)
# endif
#endif
#ifndef __nonnull
# define __nonnull(params) __attribute_nonnull__ (params)
#endif
所以，这里的宏是可以使用从1开始计数的参数位置来强制编译器检查参数不为空！神奇啊！一天学习一点就不得了了，何况学习了这么多！1. __nonnull ((1)) Attribute: Purpose: This attribute tells the compiler that the first parameter (__array) of the backtrace function must not be a null pointer. Mechanism: It leverages the __attribute__ ((__nonnull__ params)) GCC compiler extension. params: The (1) within the __nonnull ((1)) specifies that the first parameter (parameter index 1) is the one that should not be null. If there were multiple non-null parameters, you could list their indices (e.g., __nonnull ((1, 3))). 2. Macro Expansion: __nonnull(params): This macro simply expands to __attribute_nonnull__ (params). __attribute_nonnull__(params): This macro checks if the compiler supports the __nonnull__ attribute: If using GCC 3.3 or later, or if the glibc header indicates __nonnull__ support, it expands to __attribute__ ((__nonnull__ params)). Otherwise, it expands to nothing (effectively removing the attribute). 3. Compiler's Role: Compile-Time Warnings: When the __nonnull__ attribute is active, the compiler can generate warnings if it detects that the function is called with a null pointer as the first argument. Optimization: The compiler might also use this information to optimize code, as it can assume that the parameter is never null.
三月十三日 等待变化等待机会
三月十四日 等待变化等待机会
// Original function
void real_function() {
// ... implementation ...
}
// Weak alias
void old_function() __attribute__ ((weak, alias ("real_function")));
- In this example, old_function is a weak alias for real_function.
- If a user defines a strong symbol named old_function, that definition will be used.
- Otherwise, real_function will be used.
那么这样意味着我如果要使用这个功能，我需要先把巨大的模型文件搬运到那个mount里，而且还不是简单文件拷贝，这个还真的不容易啊！The Core Misunderstanding: hugetlbfs Is Not a Regular File System: This is the most crucial point. hugetlbfs is fundamentally different from regular file systems like ext4 or xfs. It's a memory-backed file system, meaning that files within it reside entirely in RAM (specifically, HugePages). You do not copy files to hugetlbfs in the same way you copy them to a disk-based file system. The Correct Usage of hugetlbfs: Allocation Within hugetlbfs: The primary purpose of hugetlbfs is to provide a mechanism for applications to allocate memory directly from HugePages. You create files within hugetlbfs to reserve and manage regions of HugePages. The files themselves do not contain data in the traditional sense; they represent memory regions. mmap and hugetlbfs: The typical usage pattern is to: Create a file within hugetlbfs with a size that matches the desired HugePages allocation. mmap that file with the MAP_HUGETLB flag. This establishes a direct mapping between the application's virtual memory and the allocated HugePages. Data Population: After mmap, you can then populate the mapped memory region with data. This data might come from another file, from network input, or be generated by the application. The important thing is that the data resides in RAM, specifically in the HugePages allocated through hugetlbfs. Why Copying Fails: No Disk Storage: hugetlbfs does not have an underlying disk storage mechanism. When you try to copy a file, the cp command attempts to write data to disk, which is not possible. Memory Allocation, Not File Copying: The kernel interprets file creation and writes within hugetlbfs as memory allocation requests. If the allocation fails (e.g., due to insufficient or non-contiguous HugePages), you'll get an error.
三月十五日 等待变化等待机会
三月十六日 等待变化等待机会
三月十七日 等待变化等待机会
static_assert(sizeof(off_t) == 8);
tgt_fd = open(argv[2], O_CREAT|O_RDWR|O_EXCL, 0666);
...
void* ptr = mmap(NULL, tgtSize, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_HUGETLB, tgt_fd, 0);
不过我目前不想去写一个while loop去把这部分读的过程完善。以后再写吧。On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are actually available right now (maybe because we were close to end-of-file, or be‐ cause we are reading from a pipe, or from a terminal), or because read() was interrupted by a signal. See also NOTES. On error, -1 is returned, and errno is set appropriately. In this case, it is left unspecified whether the file position (if any) changes.
三月十八日 等待变化等待机会
echo "# hugecp" >> README.md
git init
git add README.md
git commit -m "first commit"
git branch -M main
git remote add origin https://github.com/nickhuang99/hugecp.git
git push -u origin main
三月十九日 等待变化等待机会
Yes, the 671B model in your llama.cpp output is indeed quantized. Here's how we can tell:
File Type: The line print_info: file type = Q4_K - Medium indicates that the model has been quantized to the Q4_K format, specifically the Medium variant.
GIT_LFS_SKIP_SMUDGE=1 git clone git@hf.co:deepseek-ai/DeepSeek-R1
### 关键时间消耗点
1. **模型加载时间（主要瓶颈）**：
- 若使用 **4-bit量化**（约335GB）：
- **NVMe SSD（3.5GB/s）**：约 **2分钟**
- **SATA SSD（500MB/s）**：约 **11分钟**
- **HDD（150MB/s）**：约 **38分钟**
- 若使用 **未量化FP16模型（1.3TB）**：
- 您的内存容量（1.5TB）刚好满足，但需要更长时间加载
2. **内存初始化与预热**：
- 约 **1-5分钟**（取决于CPU并行化能力）
3. **上下文初始化**：
- 约 **30秒-2分钟**
### 总预估时间范围
- **最佳场景**（NVMe SSD + 4-bit量化）：**3-7分钟**
- **典型场景**（SATA SSD + 4-bit量化）：**12-18分钟**
- **最差场景**（HDD + FP16模型）：**40分钟+**
### 优化建议
1. **强制量化**：务必使用 `--n-gpu-layers 0` 禁用GPU加速
2. **内存预分配**：添加 `--mlock` 参数锁定内存
3. **并行加载**：设置 `--threads N`（N=物理核心数）
4. **使用mmap加速**：通过 `--mmap` 启用内存映射
三月二十日 等待变化等待机会
$ sudo numastat -p 7188
Per-node process memory usage (in MBs) for PID 7188 (llama-cli)
Node 0 Node 1 Total
--------------- --------------- ---------------
Huge 386048.00 0.00 386048.00
Heap 40.40 0.00 40.40
Stack 0.04 0.00 0.04
Private 19582.44 4.27 19586.71
---------------- --------------- --------------- ---------------
Total 405670.88 4.27 405675.15
$ numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38
node 0 size: 774008 MB
node 0 free: 318183 MB
node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
node 1 size: 774086 MB
node 1 free: 1564 MB
node distances:
node 0 1
0: 10 20
1: 20 10
void update_progress(int progress) {
int bar_length = 40; // Modify this to change the bar's length
int filled_length = (int)(bar_length * progress / 100.0);
char bar[bar_length + 1]; // +1 for the null terminator
for (int i = 0; i < bar_length; i++) {
if (i < filled_length) {
bar[i] = '=';
} else {
bar[i] = '-';
}
}
bar[bar_length] = '\0'; // Null-terminate the string
printf("\r[%s] %d%%", bar, progress);
fflush(stdout); // Ensure output is written immediately
}
$ ~/Downloads/hugecp /mnt/hugepages/deepseek-r1-numa-node0 /mnt/hugepages/deepseek-r1-no-numa
[===-------------------------------------] 8%
三月二十一日 等待变化等待机会
浅浅的clone，否则就一直结束不了。
GIT_LFS_SKIP_SMUDGE=1 git clone https://cnb.cool/ai-models/deepseek-ai/DeepSeek-R1-GGUF/DeepSeek-R1-Q8_0.git
cd DeepSeek-R1-Q8_0/
git lfs pull
这里对于这个公式要简单的理解一下，就是你每次对于下一个词的概率如果都是100%的话，那么Perplexity就是1，因为指数总和是0。而信心来自于概率大小，比如只有50%的话，总和就是1，而perplexity就是2，所以说越大信心越差啊。 另一个概念是siinusoidal position，以下是DeepSeek的解释：Perplexity is a measurement used primarily in the fields of natural language processing (NLP) and information theory to evaluate how well a probability model predicts a sample. In simpler terms, it assesses how "perplexed" or uncertain a model is when making predictions.
Key Points:
In NLP: Perplexity is commonly used to evaluate language models. It measures how well the model predicts a sequence of words. A lower perplexity indicates that the model is more confident and accurate in its predictions, while a higher perplexity suggests greater uncertainty.
Mathematical Definition: For a language model, perplexity is defined as the exponential of the cross-entropy loss. If the model assigns a probability to a sequence of words, the perplexity is:
Here, is the number of words in the sequence.
Interpretation: A perplexity of means that the model is as confused as if it had to choose uniformly and independently among possibilities. For example, a perplexity of 1 means the model is perfectly confident (ideal case), while higher values indicate worse performance.
Applications: Perplexity is widely used to compare different language models, tune hyperparameters, and assess the quality of text generation systems.
In summary, perplexity is a useful metric for understanding the performance of probabilistic models, especially in NLP, where it quantifies how well a model predicts a given sequence of words.
"Sinusoidal position" typically refers to a method of encoding positional information in sequences, commonly used in neural networks, particularly in transformer models like those in natural language processing (NLP). This technique is known as sinusoidal positional encoding.Key Points:
Purpose:
In models like transformers, the input data (e.g., words in a sentence) is processed in parallel rather than sequentially. This means the model doesn't inherently know the order of elements in the sequence.
Sinusoidal positional encoding is used to inject information about the position of each element in the sequence into the model, helping it understand the order of elements.
How It Works:
For each position in the sequence and each dimension of the embedding, a unique sinusoidal function is applied:
Here, is the dimensionality of the embedding, and is the position in the sequence.
This creates a unique pattern for each position, allowing the model to distinguish between different positions in the sequence.
Properties:
Uniqueness: Each position has a unique encoding.
Relative Position: The sinusoidal nature of the encoding allows the model to learn relative positions more easily, as the encoding for a position can be derived from the encoding for position using trigonometric identities.
Smoothness: The encoding varies smoothly across positions, which helps the model generalize better.
Applications:
Sinusoidal positional encoding is widely used in transformer-based models like BERT, GPT, and others for tasks such as machine translation, text generation, and more.
Example:
For a sequence of words in a sentence, each word's embedding is combined with its sinusoidal positional encoding to provide the model with both the word's meaning and its position in the sentence.
In summary, sinusoidal positional encoding is a technique to provide positional information to models that process sequences in parallel, ensuring they can understand and utilize the order of elements in the sequence.
- 我昨天总结了一下那篇论文，这个是和豆包的讨论，主要是我在问豆包。结果就是我认为作者的贡献在于探索并不一定开拓了一个新的实现路径，因为我觉得按照奥卡姆剃刀原则没有必要去搞一个更加复杂的模型，如果线性模型够用的话。
四月六日 等待变化等待机会
- 突然对于豆包或者gemini里为什么语音输入不工作想要探寻一下，但是首先还是要先确保麦克风是工作的，于是问豆包找到一个基本的测试网站，然后无耻的把它的代码拷贝下来给豆包做了一个最最简单的麦克风摄像头的测试网页。这个可以方便在使用视频会议中做基本测试用吧？
- 对于huggingface的封锁我开始感到气愤了，但是也许只是流量的控制，这个是无可奈何的，另一个思路是自己实现，我尝试问豆包看看，可是它还抓不住重点，仅仅停留在json文件的解析阶段，我不确定是否这么简单？
- 终于安装了豆包的chromium的浏览器插件，而且中国现在也在抵制美国的IP用户，然后语音输入就可以了。尽管不如手机的效果，也很好了。只是桌面版只有windows，我在wine里安装失败，我想是不是我要安装wine32的环境呢?
作为验证：$ WINEPREFIX=~/.wine32 wine winecfg，但是显然的豆包的安装程序是64位的，问题是你不能指望一个复杂的windows程序能够在ubuntu上运行。$ sudo apt install wine32 wine64 libc6:i386 libstdc++6:i386 $ WINEARCH=win32 WINEPREFIX=~/.wine32 winecfg- 相比于豆包，编程方面我更加信任gemini，这个算法我还是要静下心来研究看看，否则我不自己去实现的话，下载huggingface的大模型真的把我折磨死了。不知道要多少天才能下载完啊。我又补充了DeepSeek-V3的关于量化的文档，gemini有了一些小小的调整。这个看起来还是费了不少的精神，代码不麻烦，但是细节很要命，因为这么大的文件，运行下去不知道结果如何，运行起来也不知道如何。这两个python脚本是关键fp8_cast_bf16.py和kernel.py。
- 似乎huggingface使用这个形式能够极大的提高下载速度：
$ git remote -vv origin https://USER NAME:ACCESS TOKEN@huggingface.co/arcee-ai/DeepSeek-R1-bf16.git (fetch)四月七日 等待变化等待机会
- 这篇DeepSeekdeV3的技术报告我早就该读一读了。
- 下载这个bf16的DeepSeek-R1大模型花了不少钱和时间，结果我等不及了要求gemini给我写一个CPU处理的小工具。
四月八日 等待变化等待机会
- github的key折腾的够呛，可能是我的key-agent设置，也可能是需要升级加密算法，或者就是必须使用https而不是ssh，总之过程是这样子的：
似乎隔一阵子都有一些安全的机制在改变。
- 首先，还是要保证这个包的安装吧？因为ubuntu没有直接的安装包，必须这么做
原因：Unlike some older Ubuntu versions, Ubuntu 22.04 doesn't have a readily installable git-credential-libsecret package via aptsudo apt update sudo apt install libsecret-1-0 libsecret-1-dev libglib2.0-dev build-essential cd /usr/share/doc/git/contrib/credential/libsecret sudo make git config --global credential.helper /usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret- 最好是按照github的要求使用高级算法：
然后在github的ssh-key页面把~.ssh/id_ed25519.pub拷贝到github上。$ ssh-keygen -t ed25519 -C "your_email@example.com" $ eval "$(ssh-agent -s)" ssh-add ~/.ssh/id_ed25519- 注意这里repo的remote url不能是ssh格式，我不知道为什么，只能是https:
$ git remote set-url origin https://github.com/{USERNAME}/{REPONAME}.git $ git push origin- 我耗费了一百多美金的流量下载的bf16的格式似乎不完全，但是git lfs pull直接就返回，这个真的是头疼！
- 所以，凡事都不能在一棵树上吊死，我的planB就是自己写一个小工具来转换，这个实际上是gemini写的，我改的，当然设计改进还是我的。 但是这个还需要使用llama.cpp的convert_hf_to_gguf.py来再转换为gguf格式，它需要至少
等等配置才能工作。这个代码以后肯定是要改进的，不过现在先使用一下看看行不行。
- free -h显示有很多的内存被used，但是又是available，所以，应该是在cached，可以使用
之所以要做这个是因为我使用sysctl vm.nr_hugepages=1350无法分配更多的pages，因为只看到/proc/meminfo里只有70个，这个是hugepagesz=1G的情况。$ echo 1 | sudo tee /proc/sys/vm/crop_caches- 为什么sysctl立刻就返回了，但是hugepages实际上并没有分配呢？
所以，大致上这个就是和内核通讯的方式。$ cat /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/free_hugepages 282 $ echo 400 |sudo tee /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages 400 $ cat /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages 282四月九日 等待变化等待机会
- 我自己转的模型也许有错误，也许是内存不够，因为bf16的DeepSeek-R1:671B文件/内存需要1.3T，这个可能是超过了我的上限了，总之输出不正常。这个是另一个思路，在aws/s3上存储，首先使用s3fs来创建一个mount point。
我发现我的aws/ec2的磁盘不能把cache放在/tmp上因为它是一个tmpfs只有几十k。只有在根目录还有几个G的空间。我为了防止磁盘被耗尽强制预留2000MB的磁盘空间。 这里必须把密码文件放在${HOME}/.aws/credentials只不过这里的${HOME}是root$ sudo s3fs www.staroceans.org:/DeepSeek /mnt/s3-repo/ -o allow_other -o use_path_request_style -o default_acl=public-read -o use_cache=/cache -o parallel_count=10 -o ensure_diskfree=2000 -o multipart_size=50- 我决定放弃满血DeepSeek，转而使用蒸馏版的小模型。那么笔记本上运行要快的多，只不过我又要安装驱动。
gemini建议还是安装nvidia-driver-570 ，但是这个一定会安装dkms，然后内核又要被修改，我发现它是把我的内核降级了，这个实在是讨厌，很多问题就出在这个内核版本改变的问题上。而且我还是最好重启再安装cuda。 我问DeepSeek是否必须要安装cuda，它说：$ ubuntu-drivers devices == /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 == modalias : pci:v000010DEd000028A1sv00001558sd0000A650bc03sc00i00 vendor : NVIDIA Corporation driver : nvidia-driver-535-server-open - distro non-free driver : nvidia-driver-535-server - distro non-free driver : nvidia-driver-560-open - third-party non-free driver : nvidia-driver-560 - third-party non-free driver : nvidia-driver-565-open - third-party non-free driver : nvidia-driver-565 - third-party non-free driver : nvidia-driver-570 - third-party non-free recommended driver : nvidia-driver-535-open - third-party non-free driver : nvidia-driver-545-open - distro non-free driver : nvidia-driver-535 - third-party non-free driver : nvidia-driver-570-server - distro non-free driver : nvidia-driver-525 - third-party non-free driver : nvidia-driver-550 - third-party non-free driver : nvidia-driver-550-open - third-party non-free driver : nvidia-driver-570-server-open - distro non-free driver : nvidia-driver-555 - third-party non-free driver : nvidia-driver-570-open - third-party non-free driver : nvidia-driver-545 - third-party non-free driver : nvidia-driver-555-open - third-party non-free driver : xserver-xorg-video-nouveau - distro free builtin
Option 1: Install Minimal CUDA Libraries (Recommended)
- install
sudo apt install nvidia-cuda-toolkit- verify
dpkg -L nvidia-cuda-toolkit | grep libcublas- build
cd llama.cpp make clean make LLAMA_CUBLAS=1Option 2: Use OpenCL/CLBlast (No CUDA)
- Install OpenCL libraries:
sudo apt install nvidia-opencl-dev ocl-icd-opencl-dev clinfo- Verify OpenCL detection:
clinfo # Check if your GPU appears- Build llama.cpp with OpenCL:
cd llama.cpp make clean make LLAMA_CLBLAST=1- Run with OpenCL:
./main -m <model> -n 128 -ngl 50 --gpu-layers 50Which Should You Choose?
Approach Pros Cons Minimal CUDA Faster performance, easy setup Still requires CUDA libraries OpenCL Avoids CUDA entirely Slower, less optimized for NVIDIA GPUs 四月十日 等待变化等待机会
- 看了llama.cpp的模型解析机制才明白它的地层就是所谓的gglm的库的工作。这里我想来比较一下1.5b大模型和7b大模型的区别。 那么差别是什么呢？ 去除无关的部分列表如下：
item 1.5b 7b qwen2.embedding_length 1536 3584 qwen2.feed_forward_length 8960 18944 qwen2.attention.head_count 12 28 qwen2.attention.head_count_kv 2 4 tokenizer.ggml.tokens arr[str,151936] arr[str,152064] tokenizer.ggml.token_type arr[i32,151936] arr[i32,152064] file size 1.04 GiB (5.00 BPW) 4.36 GiB (4.91 BPW) n_embd 1536 3584 n_head 12 28 n_head_kv 2 4 n_gqa 6 7 n_embd_k_gqa 256 512 n_embd_v_gqa 256 512 n_ff 8960 18944 model params 1.78 B 7.62 B n_vocab 151936 152064 CPU_Mapped model buffer size 1059.89 MiB 4460.45 MiB CPU KV buffer size 112.00 MiB 224.00 MiB KV self size 112.00 MiB, K (f16): 56.00 MiB, V (f16): 56.00 MiB 224.00 MiB, K (f16): 112.00 MiB, V (f16): 112.00 MiB CPU compute buffer size 299.75 MiB 304.00 MiB 四月十三日 等待变化等待机会
- 简单的看看DeepSeek-R1的权重发现几乎没有一个byte是0的，这个让人吃惊，可是也是在意料之中吧？早先那些针对GPT的大模型里都是0的权重肯定都是过时的想法了。说不定DeepSeek还是某种加
密型式，我这里的密是紧密的意思，因为i所有权重都找不到一个0的情况不是很特殊吗？- gguf如何存储各种各样的数据的？ 这里就是这么几种情况
- 简单类型放在data里，反正是一个万能的容器。
- 某种类型的数组，也是放在data里，那么如何取出来呢？这里有一个关键就是如何存放类型了，这个是经典的c++的解决办法，因为传统的c程序员只能存放类型的长度，无法对于类型动态的存储，于是就有混淆，比如都是4个byte的数字到底是浮点数还是整数呢？这个是经典的C++模板的威力，首先定义个万能模板，就是所谓的general的类型 然后我们依据不同类型的特例化 不过美中不足的是作者使用一个map来存储长度，我本来想着在存储的时候自动把类型的长度写下来存储起来，但是各有利弊。
- llama的vocabulary也就是llama_vocab，它要看这些属性：LLM_KV_TOKENIZER_MODEL和LLM_KV_TOKENIZER_PRE以及它的数量LLM_KV_TOKENIZER_TOKEN_TYPE_COUNT 而DeepSeek-R1的模型类型是gpt2，之前的类型是qwen2，这个模型需要查看LLM_KV_TOKENIZER_MERGES，而这里的所谓的merge是存储的一对用空格分开的所谓的
分词或者称之为subword。于是我们的词汇表实际上是存储为一对分词。怎么使用呢？但是我们第一步还到不了这里因为要先把UTF-8变为所谓的unicode。这个就是unicode_cpts_from_utf8，这个代码是比较经典的，因为没有什么更好的做法，都是一个最多四个byte的转换，当然要去判断第一个byte的长度bit，我在很多地方看到过，也没有听说有什么更高效的办法，也许将来芯片可以去做吧？
- 针对用户输入，首先要使用regex来过滤unicode_regex_split，而每一个大模型的tokenizer都会附带一个regex来
- llm_tokenizer_bpe是llama内部存储的regex，但是照例说这个应该是从safetensors一起配套的tokenizer.json里读到的，但是似乎llama存了一个固定的，而且有好几个版本，这个是
PRE_TYPE_DEEPSEEK_LLM这个是PRE_TYPE_DEEPSEEK3_LLM这个是PRE_TYPE_DEEPSEEK_CODER我们来看看t在DeepSeek-R1的tokenizer.json里怎么写的吧？ 换言之，DeepSeek-R1是从DeepSeek-V3继承来的，这个当然是废话。也就是说照例说应该是使用convert_hf_to_gguf.py直接从tokenizer.json里面拷贝来的，而llama.cpp是直接拷贝到代码里的，这个应该是大模型的核心紧密的部分，没法改变，因为token是核心，每个模型稍微变一点点就鸡同鸭讲了，这个当然也是废话。- 针对regex里有所谓的
collapsed的做法，代码里是看这个表达式是否在regex里出现： 很显然的DeepSeek-R1有这个"\\p{N}{1,3}"，所以是collapsed。这里是关于它的讨论，说老实话我一眼看过去是看不明白的，但是我知道这里的水很深，非常的麻烦，尤其我至今也搞不明白表达式本身是否应该使用unicode，比如中文你要怎么表达呢？而且还有跨平台，因为似乎C++的实现也有支持的问题，以前听说std的实现不如boost的，有重大的效率问题等等。总而言之是很复杂的问题。这里大侠使用工具脚本从unicode官方网站下载了这个表。我的感觉是所有的unicode的codepoints，然后脚本把合法的分段，因为不连续吧？更加有用的是把unicode进行了分类，到底是数字还是字符还是标点等等。这个是非常繁琐的工作，但是你也没有什么好办法。The original unicode text is converted to a single-byte extended ASCII, using the following transformation: I.e., all single-byte codepoints less than 128 remain the same, while the rest of the codepoints are converted (collapsed) to one of the following values: 0xD0, 0xD1, 0xD2, 0xD3 depending on their codepoint category: Next, the category patterns in the original regex are replaced to match the set of the collapsed byte + the sub-128 codepoints from that category: For example, the regex \p{N} becomes [\xD1\x30-\39] We now apply the "collapsed" regex onto the "collapsed" string using std::regex to get the splitting offsets This algorithm only works if there are no non-ASCII characters in the original regex. For example, we cannot apply this to: "[一-龥ࠀ-一가-]+|\\p{N}+", but luckily there seem to be no such patterns. Regex use either unicode characters or unicode categories but never both If a regex contains non-ASCII characters and does not contain unicode categories, we apply std::wregex- 所以，一言以蔽之说是一个简单的说要把unicode_cpts_from_utf8但是里面的工作很多，这里还有模型自身的要求。这里unicode_regex_split_custom_gpt2是模型自身要求的regex，难道说大侠把regex实现了一遍？应该说是针对早期的特殊简单regex的特例实现了一下，后来各个大模型千奇百怪的regex怎么可能把轮子都造一遍呢？但是很关键的是如何解决所谓的std::regex / std::wregex，我曾经尝试过一次非常的混乱，似乎有大侠说不要使用后者，可是你能避免吗？
- 其实最终的目的是什么？是根据模型自带的regex来做所谓的
分词，我的理解就是每个模型训练的时候是针对特定语言的，而且是特别的分词来训练的，所以，这个来不得半点差错，这个当然是废话。所以，我们在使用模型自带的regex得到这个函数unicode_regex_split_stl的返回的一个所谓的bpe_offset就是一个分词的偏移向量或者说数组，我自然的想那就是要做merge的了。- 我现在开始怀疑DeepSeek的regex是不是正确，因为各个online的regex网站都认为它不对。 这个就是很简单的任何一个除了下划线_以外的标点符号跟着无限长的字符。再次牢记在character classes或者说方括号里只有这几个特殊字符[]\-^，而且不允许有[]递归。就是说[]是顺序的。如果你在代码里不使用Raw的话肯定是很难受的。
- 折腾了好久分别拷问gemini和deepseek，总算有些开窍了，这个\p{N}{1,3}是指的unicode被划分为数字的字符，而这个不包含一二三四五六七八九十，它们是所谓的Lo，letter of others
四月十四日 等待变化等待机会
- 昨天是头脑极度混乱的状态下，因为每走一步都遇到很多问题导致不断的偏离轨道去解决旁枝末节的基础。任何搞科研和工程实践中如果基础不够雄厚都是如此，因为你会在所有的细节上暴露自己的无知与愚蠢，然后你的能力体现在以多快的速度从这些被牵扯的问题里拉回来。虽然不是所有的细枝末节都应该要解决的，可是有很多暴露出的问题是核心，一旦被搁置也许导致的后果是不可预测的。
- 我现在回想起来应该继续看tokenzier的merge如何被使用，这个原理我在llama.c那一页代码实现transformer里就看到过，只不过当时不敢相信。现在我们来证实一下原理吧。
- 但是我依然需要完成regex里的重要的unicode分类的问题，这个是gemini的解答：
Common Unicode Categories
L: Letter Matches any kind of letter from any script. Examples: a, A, あ, 漢字 P: Punctuation Matches any kind of punctuation character. Examples: ., ,, !, ?, ", ' S: Symbol Matches any kind of symbol. Examples: $, %, +, ©, ★ M: Mark Matches a character that combines with or modifies another character. Examples: Accents, diacritics (e.g., in á, è, or combining characters in some Asian scripts).Full List of General Categories
L: Letter Lu: Uppercase Letter Ll: Lowercase Letter Lt: Titlecase Letter Lm: Modifier Letter Lo: Other Letter M: Mark Mn: Nonspacing Mark Mc: Spacing Combining Mark Me: Enclosing Mark N: Number Nd: Decimal Digit Nl: Letter Number No: Other Number P: Punctuation Pc: Connector Punctuation Pd: Dash Punctuation Ps: Open Punctuation Pe: Close Punctuation Pi: Initial Punctuation Pf: Final Punctuation Po: Other Punctuation S: Symbol Sm: Math Symbol Sc: Currency Symbol Sk: Modifier Symbol So: Other Symbol Z: Separator Zs: Space Separator Zl: Line Separator Zp: Paragraph Separator C: Other Cc: Control Cf: Format Co: Private Use Cs: Surrogate- 在明白了category之后再来审视DeepSeek的regex，就感觉不是很能理解，我去除了多余的escape之后，再把三组合在一起：
- 这个太复杂，我们再拆分一下，根据regex101.com的提示：
- 1st Alternative 这里揭示出我的理解错误在character class或者说[]之间
Negation (所以，^_没有特殊含义。 这些特殊字符就是本身再加上至少一个英文字母。^): As you mentioned, inside a character class,^has a special meaning only when it's the very first character after the opening square bracket[. In that position, it negates the character class (meaning "match any character not in this set"). If^appears anywhere else within the character class, it's treated as a literal^character.- 2nd Alternative 这个现在也比较清楚了，只多一个开头的非换行符，非字母，非标点，非符号，然后跟随至少一个字母或者控制符(mark)
- 3rd Alternative 注意第一个空格字符
matches the character with index 32102016 or408) literally (case sensitive)gemini替DeepSeek辩护反正我是不信。就是至少一个空格带着若干标点或者字母以及若干回车符Why Not Just \s? You'd think that \s would be the most straightforward way to represent a space in a regex, and often it is. However, there can be subtle reasons or contexts where a literal space is used, or where there might be a need to be very specific: Clarity in a Specific Context: In some cases, a regex might be written to be extremely explicit, especially if it's dealing with a situation where other whitespace characters (like tabs or newlines) could cause problems. Using a literal space ensures that only a space character is matched. Potential for \s Redefinition: In some very specific or unusual regex environments, the definition of \s might be altered or extended to include other characters beyond the standard whitespace set. While rare, this could lead to unexpected behavior if you rely on \s when you truly only want a space. Historical Reasons/Legacy: Some older regex patterns or systems might have been written before Unicode support and the modern understanding of whitespace were fully standardized.- 4th Alternative 这个应该是无争议的吧？就是一堆无意义的空白字符
- 5th Alternative 白字符加上非非白字符，那是什么？
\S matches any non-whitespace character (equivalent to [^\r\n\t\f\v \u00a0\u1680\u2000-\u200a\u2028\u2029\u202f\u205f\u3000\ufeff])gemini说关于?!我很久以前用过又要补课了(?!\S): This is a negative lookahead assertion. It asserts that what immediately follows the matched whitespace is not a non-whitespace character. In other words, it asserts that the whitespace is followed by either the end of the string or another whitespace character.Okay, I'll explain what
?!means in regex, referring to the information in the Canvas document, "Regex with \s for Space Character".
?!in regex is used for a negative lookahead assertion.Here's a more detailed explanation:
- Lookahead Assertion: A lookahead assertion is a type of zero-width assertion. This means it doesn't consume any characters in the string being searched; instead, it asserts a condition about what comes after the current position.
- Negative Lookahead (
?!): The negative lookahead assertion(?!...)asserts that the expression inside the parentheses cannot be matched after the current position.In simpler terms,
(?!...)checks if a certain pattern does not follow the current position in the string. If the pattern does follow, the match fails. If the pattern does not follow, the match succeeds.- 东亚文字(只有中日没有韩国，韩文不算东亚文字，纯粹是自创的脱离中华文化圈的蛮夷文化)(以上解读不代表我的观点纯粹是DeepSeek-R1的regex内涵)
- 一到三个数字
- 总而言之，DeepSeek的regex没有意义，我从llama.cpp里扣出来的似乎才对
四月十五日 等待变化等待机会
- 终于明白大神为什么费劲实现这个Unicode Category了，因为C++的regex不支持啊！只能自己做！这个是DeepSeek的解释：
The C++ Standard Library's对于这个\p{N}失败的原因是std::regexengine (as defined in<regex>) does not natively support Unicode property escapes like\p{N}(for numeric characters) or other Unicode categories (e.g.,\p{L}for letters). This is a known limitation of the default regex syntax in C++, which is based on ECMAScript syntax (the default grammar) and lacks Unicode property support.WhyDeepSeek给出了根本上语法就没有实现。所以，要么自己实现要么依赖第三方库。我这才明白为什么自己去创建那个UnicodeData.txt的表，这个是没有办法啊。\p{N}Fails
The
\p{...}syntax is part of Unicode Technical Standard #18 (UTS18) and is supported by regex engines like PCRE, ICU, or Perl, but not by the C++ Standard Library'sstd::regex.In ECMAScript-compatible mode (the default for
std::regex),\pis interpreted as a literalp, and{N}is parsed as an invalid range quantifier (e.g.,{min,max}), leading to the "invalid range" error.- 关于DeepSeek自带的regex和llama.cpp修改的版本
差异对比是这样子的 一模一样！连那个容易被忽略的空格符都是一样的。那么我为什么怀疑它的正确性呢？原因当然是其中的意义在哪里？当然首先我问了DeepSeek和Gemini如果同样的这个对于python和C++的regex引擎表现是否不同，gemini认为差别主要就是UnicodeCategory的不支持上。- 类似的在www.regex101.com上不能选择ECMAScript，这样子结果似乎是对的，但是regex101.com也许实现有瑕疵吧？就是它的python的falvor对于\\-在普通python字符串里依然不认为它是被转义的。这个已经远远的偏离了我的目的。就是说如果我使用不受代码语言escape干扰的regex在PCRE2下能够正确覆盖所有的匹配我就应该是可以信服DeepSeek的regex没有问题了，至于说我在我本地的python脚本里测试不能达到所有的预期是gemini说的。我不妨问问DeepSeek交叉验证一下。结果DeepSeek太牛了，它不但会生成更加详细的测试例，而且有agent功能就是直接运行python代码把结果都分析了而且还提出改正的结果以及改正后的测试，我已经彻底被替代了，因为之前我还是一个工具人执行拷贝粘贴代码及其运行结果的
手的功能，现在彻底被代替了。- 对于非python程序员来说你写一个例子命名为regex.py就会导致什么问题呢？无限递归的import啊！不过很多错误到底是python的还是表达式的，我也不知道。这个重要吗？
四月十九日 等待变化等待机会
- 不论一个算法写的有多么清晰与详细，对于没有编程经验的人来说依然是天书，类似的，哪怕是略有经验的，一知半解的也依然百思不得其解。首先是问题的理解，如果你压根不明白问题在哪里，那你压根儿不应该看这个代码。对于实践者来说，为了部署与平台一致性不愿意使用boost的regex是可以理解的。毕竟你不希望任何一个编程小白为了能够使用ollama而安装boost的regex的动态库吧？哪怕是部署静态库也是不可接受的，也许还有效率的考虑。总而言之，因为std::regex对于unicode的支持不充分，就是unicode category没有实现，导致大神费劲周折想出了这么一个相当的复杂的算法。我看了差不多一个星期才大概理解了吧？
- 首先是所谓的
collapsed英文我始终没有真正的理解，对于一个Unicode的category比如\p{N}要怎么理解？这个是只有在Unicode下才能来起作用的category，那么照理说你应该交给regex来处理，可是regex没有实现，怎么办？我能否简单的用[0-9]替代呢？这个应该是我的直觉，看起来是诱人的，因为如果我把regex的pattern也从UTF-8的编码或者ASCII的单字节转为Unicode，也就是说我不使用regex而是使用wregex来处理已经从UTF-8编码转为Unicode的输入，这样子不是就可以了吗？这里也许还有很多的不确定性，但是大神走的是另一条路。不是膨胀而是收缩，就是把unicode缩减为单字节。比如费心费力的把Unicode所有的字符都制作了一个表，当然是把所有的code都分类成{L,P,S,M,N}这么几大类，这个实际上就是实现了原始的分类。可是关键的是细节，我大概都能猜出一些路径，但是依旧卡壳了。因为归根结底你这么做并不能解决复杂的regex的pattern究竟想要干什么，你到底是要这些还是不要这些在表达式里是千奇百怪的，而且作为一个character class的表达式是不允许嵌套的，这就是大神费劲的自己去解析原版的regex来查看这个\p{L}之类的是否是在character class内，就是在[]内，而最最关键的一点，我却没有理解到，因为大神使用了看似多余的做法，替换regex pattern的\p{N}的动作是双重的，既使用等效表达式[0-9]，又使用自己定义的category的符号标志，而这个就是迷惑我的地方，直到我使用gdb看运行期数据才恍然大悟，其实，你把unicode缩减为五大类的标志位，那么对于class category它们就是literal了，而这里大神为了不知名的优化刻意把ASCII的unicode对应部分免除了这个流程导致了一个现象，就是ASCII和自定义的这五大类的标志并行。 如此复杂和未充分优化的算法让我要吐血了。因为我自己也不会明白我写的是什么。因为当最后把Unicode的category进行替换之后，你会看到还有之前过滤掉的ASCII混合在一起，这就是为什么最后又要反复调用unicode转为UTF-8的原因。这里我的疑惑就是难道大模型的regex语法是只针对unicode吗？我现在累死了想不清楚了。四月二十一日 等待变化等待机会
- 有些最最基本的我却缺乏概念！什么是wchar_t，这个简直是让人汗颜啊！我自己存的都是吃灰的从来没有用过啊！
wchar_t — type for wide character representation (see wide strings). It has the same size, signedness, and alignment as one of the integer types, but is a distinct type. In practice, it is 32 bits and holds UTF-32 on Linux and many other non-Windows systems, but 16 bits and holds UTF-16 code units on Windows. The standard used to require wchar_t to be large enough to represent any supported character code point. However, such requirement cannot be fulfilled on Windows, and thus it is considered as a defect and removed.我终于明白我为什么记不住这个wchar_t因为在Windows下它是有缺陷的所以弃而不用，这个算是给我找了一个记不住的理由！- 这个ASCII码表本来应该是熟记于心的但是总是有几个控制字符记不住：
ASCII values characters decimal hexadecimal octal 0–8 \x0–\x8\0–\10control codes ( NUL, etc.)≠0000000000009 \x9\11tab ( \t)≠00≠0≠00000000010–13 \xA–\xD\12–\15whitespaces ( \n,\v,\f,\r)≠00≠000000000014–31 \xE–\x1F\16–\37control codes ≠00000000000032 \x20\40space 0≠0≠0≠00000000033–47 \x21–\x2F\41–\57!"#$%&'()*+,-./0≠000≠0≠000000048–57 \x30–\x39\60–\7101234567890≠000≠00≠0000≠0≠058–64 \x3A–\x40\72–\100:;<=>?@0≠000≠0≠000000065–70 \x41–\x46\101–\106ABCDEF0≠000≠00≠0≠0≠000≠071–90 \x47–\x5A\107–\132GHIJKLMNOPQRSTUVWXYZ0≠000≠00≠0≠0≠000091–96 \x5B–\x60\133–\140[\]^_`0≠000≠0≠000000097–102 \x61–\x66\141–\146abcdef0≠000≠00≠0≠00≠00≠0103–122 \x67–\x7A\147–\172ghijklmnopqrstuvwxyz0≠000≠00≠0≠00≠000123–126 \x7B–\x7E\172–\176{|}~0≠000≠0≠0000000127 \x7F\177backspace character ( DEL)≠000000000000- 在古老的C程序员习惯里声明一个数组并初始化往往就是最简单的工作，但是不定长度要怎么做呢？就空白吧，可是这个是一个好习惯吗？它毕竟是一个运行期的东西，为什么不使用std::initializer_list
当然我似乎找不出一定要的理由，比如An object of type std::initializer_listis a lightweight proxy object that provides access to an array of objects of type const T (that may be allocated in read-only memory). C程序员是可以做到使用std::begin(arr)，当然这里的constexpr并不是必须的，因为const是足够的，只不过我要使用static_assert，来证明编译必须通过性而已。const int arr[] = {1,2,3,4,5}; static_assert(sizeof(arr) == sizeof(int)*5, "sizeof array");constexpr int arr[] = {1,2,3,4,5}; static_assert(sizeof(arr) == sizeof(int)*5, "sizeof array"); static_assert(*std::begin(arr) == 1, "arr[0] == 1");- 大侠的注释里说wregex不能识别\s
代码里这些所谓的whitespace是这么定义的：// std::wregex \s does not mach non-ASCII whitespaces, using 0x0B as fallback std::wstring wtext(cpts.begin(), cpts.end()); for (size_t i = 0; i < wtext.size(); ++i) { if (wtext[i] > 0x7F && unicode_cpt_flags_from_cpt(wtext[i]).is_whitespace) { wtext[i] = 0x0B; } }可是根据Unicode的定义表： 但是看大侠的脚本的注释才找到这个文档： 这个我将信将疑。这里的关键是在调用std::wcout之前和之后都要调用fwide(stdout, -1); 才能保证随后的std::cout能够使用窄字符了。 结果呢？从regex的角度来看，这个是无厘头啊！有很多人认为\s应该包含这些所谓的控制符。 注意\u0085这个whitespace不被承认，这个就是大侠所做的，而我花了一个早上才搞明白！还有很多其他的不被承认的whitespace字符。const std::unordered_setunicode_set_whitespace = { 0x000009, 0x00000A, 0x00000B, 0x00000C, 0x00000D, 0x000020, 0x000085, 0x0000A0, 0x001680, 0x002000, 0x002001, 0x002002, 0x002003, 0x002004, 0x002005, 0x002006, 0x002007, 0x002008, 0x002009, 0x00200A, 0x002028, 0x002029, 0x00202F, 0x00205F, 0x003000, }; $ ./test7 about to process:space ,nextline ,linetab ,linefeed ,tab !NEL\u0085�!rriage with regex:\s [0x20] [0xa] [0xb] [0xc] [0xd] [0x9] about to process:space ,nextline ,linetab ,linefeed ,tab !NEL\u0085�!rriage with regex:\s+ [0x20] [0xa] [0xb] [0xc] [0xd] [0x9] about to process:space ,nextline ,linetab ,linefeed ,tab !NEL0x85�!carriage with regex:\s [0x20] [0xa] [0xb] [0xc] [0xd] [0x9] about to process:space ,nextline ,linetab ,linefeed ,tab !NEL0x85�!carriage with regex:\s+ [0x20] [0xa] [0xb] [0xc] [0xd] [0x9] nick@nick-sager:/tmp$- 一个早上都在为一个个拦路虎所困扰，有些是白痴问题，比如这个gemini解释的很好
- The form feed (
\f) likely moved the cursor to the beginning of a new line.- The carriage return (
\r) moved the cursor back to the beginning of the current line (the one where the comma was printed after the form feed).- The subsequent characters "tab" overwrote the beginning of that line.
- The tab (
\t) then moved the cursor further to the right before printing the "!".- The characters "carriage" were printed before the carriage return moved the cursor back.
- 但是这个问题呢？ 这个问题要怎么解释呢？
- 提了一个GCC的bug。问题就在于这个明显是库的问题，因为MSVC是正确的。
- 提完了bug，我都快累死了。一个早上就在这个泥潭里打转转，我都彻底忘记要干什么了。哦，我大概明白了大神所说的wregex的\s仅仅支持ASCII码的whitespace的意思了，就是说没有问题只要是在127以内的这些所谓的whitespace是可以抓到的，但是超过部分就不行了！
- 偶然看到以前的一个文档，这个应该对于我现在的服务器上的RAID卡是有帮助的。
四月二十三日 等待变化等待机会
- 一个貌似很简单的问题，但是就是解决不了，让我怀疑OpenCV究竟能做什么？
- 奋战了一个晚上，终于gemini完成了我的期待，我把代码存放在这里。这个中间修正了多少bug呢？简直是难以计数，我中间都几乎放弃了好几次。太累了。
四月二十六日 等待变化等待机会
- 这个编程的强度还是很大的，尽管我自己几乎没有怎么写代码，可是我的工作强度还是很大的，因为我也需要运行debug还有有时候必须看看gemini的代码，自从它上一次胡说八道SGF之后我还是要对它小心一些，否则最后带到沟里去也不知道，所幸所有的问题都在测试中呈现。这句是计算机世界里很难造假的原因，太容易发现了，因为能够重现的假象一定很难造。如果真的造成了，那很可能是真相。总而言之，关于图像处理的稳定工作我想放一下，等到真的围棋棋盘到了再说，实际的测试环境也许更容易，因为一开始的棋盘是简单的，只有中盘才是复杂的。
- 针对ubuntu下存取webcam的照片访问/dev/videoX和使用ioctl的权限问题，我实在是不原因使用sudo，因为太危险，而且debug环境也有问题，一个简单的解决办法就是把我自己加入到video这个group里：
当然这样做你要明白一件事就是在调用webcam的v4l2接口它应该是使用所谓的effective user id吧，总而言之，你是在video这个group下，那么本地的文件读写又成了问题，我本来以为video也是一个用户，才发现它只是一个group，那么我只能把文件输出到/tmp或者预先设定了other写权限的地方。$ sudo usermod -a -G video $(id -nu)四月二十七日 等待变化等待机会
- 老爸的《卜算子》
《卜算子》春夏之交 云雾谷雨天， 芳菲春无力。 杜鹃啼叫布谷声， 催播田种地。 ***** 旧叶换新芽， 花谢结果子。 原野翠绿雨乍晴， 夏来南风起。我修改了一下：《卜算子--春夏之交》 云雾谷雨天， 春去芳菲寂。 杜鹃啼血布谷寒， 催人耕田地。 ***** 旧叶换新芽， 花谢结青子。 原野翠绿雨乍晴， 风暖夏将至。- 这个是标准的做法
感觉gemini的代码生成过程有重大问题，就是说它的code diff总是出错。#include#include #include class GEMError : public std::runtime_error { public: GEMError(const std::string &message, const char* file, int line, const char* function) : std::runtime_error(constructMessage(message, file, line, function)) {} private: static std::string constructMessage(const std::string &message, const char* file, int line, const char* function) { std::stringstream ss; ss << message << " in " << function << " at " << file << ":" << line; return ss.str(); } }; // Macro for throwing GEMError exceptions with source code information #define THROWGEMERROR(message) \ throw GEMError(message, __FILE__, __LINE__, __PRETTY_FUNCTION__); class SGFError : public std::runtime_error { public: SGFError(const std::string &message) : std::runtime_error(message) {} };
