规整的指令编码。RISC-V的指令集编码非常的规整，指令所需的通用寄存器的索引（Index）都被放在固定的位置。因此指令译码器（Instruction Decoder）可以非常便捷的译码出寄存器索引然后读取通用寄存器组（Register File，Regfile）。

RISC-V最基本也是唯一强制要求实现的指令集部分是由I字母表示的基本整数指令子集，使用该整数指令子集，便能够实现完整的软件编译器。其他的指令子集部分均为可选的模块，具有代表性的模块包括M/A/F/D/C。

S-Mode虚存的地址转换 S、U-Mode中虚拟地址会以从根部遍历页表的方式转换为物理地址： satp.PPN 给出了一级页表基址， VA [31:22] 给出了一级页号，CPU会读取位于地址(satp. PPN × 4096 + VA[31: 22] × 4)页表项。 PTE 包含二级页表基址，VA[21:12]给出了二级页号，CPU读取位于地址(PTE. PPN × 4096 + VA[21: 12] × 4)叶节点页表项。 叶节点页表项的PPN字段和页内偏移（原始虚址的最低 12 个有效位）组成了最终结果：物理地址(LeafPTE.PPN×4096+VA[11: 0])

satp（Supervisor Address Translation and Protection，监管者地址转换和保护）S模式控制状态寄存器控制分页。satp 有三个域： MODE 域可以开启分页并选择页表级数 ASID（Address Space Identifier，地址空间标识符）域是可选的，避免了切换进程时将TLB刷新的问题，降低上下文切换的开销 PPN 字段保存了根页表的物理页号

mcause CSR寄存器 当发生异常时，mcause CSR中被写入一个指示导致异常的事件的代码，如果事件由中断引起，则置上Interrupt位，Exception Code字段包含指示最后一个异常的编码

mstatus(Machine Status)保存全局中断以及其他的状态 SIE控制S-Mode下全局中断，MIE控制M-Mode下全局中断。 SPIE、MPIE记录发生中断之前MIE和SIE的值。 SPP表示变化之前的特权级别是S-Mode还是U-Mode MPP表示变化之前是S-Mode还是U-Mode还是M-Mode

PP：Previous Privilege

M-Mode的中断控制和状态寄存器 mtvec(MachineTrapVector)保存发生中断/异常时要跳转到的中断处理例程入口地址 mepc(Machine Exception PC)指向发生中断/异常时的指令 mcause(Machine Exception Cause)指示发生中断/异常的种类 mie(Machine Interrupt Enable)中断使能寄存器 mip(Machine Interrupt Pending)中断请求寄存器 mtval(Machine Trap Value)保存陷入(trap)附加信息 mscratch(Machine Scratch)它暂时存放一个字大小的数据 mstatus(Machine Status)保存全局中断以及其他的状态

中断/异常开销 建立中断/异常/系统调用号与对应服务的开销； 内核堆栈的建立； 验证系统调用参数； 内核态数据拷贝到用户态； 内存状态改变（Cache/TLB 刷新的开销）。

内存状态改变（Cache/TLB 刷新的开销）

RISC-V 要求实现精确异常：保证异常之前的所有指令都完整执行，后续指令都没有开始执行

Risc-V中异常和中断统称Trap

CSR寄存器功能 信息类：主要用于获取当前芯片id和cpu核id等信息。 Trap设置：用于设置中断和异常相关寄存器。 Trap处理：用于处理中断和异常相关寄存器。 内存保护：有效保护内存资源

OS通过硬件隔离手段（三防）来保障计算机的安全可靠 设置 CSR(控制状态寄存器) 实现隔离 权力：防止应用访问系统管控相关寄存器 地址空间配置寄存器：mstatus/sstatus CSR(中断及状态) 时间：防止应用长期使用 100％的 CPU 中断配置寄存器：sstatus/stvec CSR（中断跳转地址） 数据：防止应用破坏窃取数据 地址空间相关寄存器：sstatus/stvec/satp CSR （分页系统）

控制状态寄存器(CSR：Control and Status Registers) 通过控制状态寄存器指令访问，可以有4096个CSR 运行在用户态的应用程序不能访问大部分的CSR寄存器 运行在内核态的操作系统通过访问CSR寄存器控制计算机

M-Mode（Machine Mode, Physical Machine Mode） 特权级模式：控制物理内存，直接关机 是Bootloader/BIOS运行的Machine Mode CPU执行模式 能执行M-Mode特权指令，能直接影响上述其他软件的执行

H-Mode(Hypervisor Mode, Virtual Machine Mode，虚拟机监控器) 特权级模式：限制OS访问的内存空间 是虚拟机监控器运行的Hypervisor Mode CPU执行模式，能执行H-Mode特权指令，能直接影响OS执行

S-Mode（Supervisor Mode, Kernel Mode，内核态，内核模式） 在内核态的操作系统具有足够强大的硬件控制能力 特权级模式（Privileged Mode）：限制APP的执行与内存访问 是操作系统运行的内核态CPU执行模式 能执行内核态特权指令，能直接影响应用程序执行

U-Mode （User Mode，用户模式、用户态） 非特权级模式（Unprivileged Mode）：基本计算 是应用程序运行的用户态CPU执行模式 不能执行特权指令，不能直接影响其他应用程序执行

M Mode：小型设备（蓝牙耳机等） U+M Mode:嵌入式设备（电视遥控器、刷卡机等）

如果是正常推出，uCore 会自动关闭 qemu，但如果 os 跑飞了，我们不能通过 Ctrl + C 来推出。此时可以先按下 Ctrl+A ，再按下 X 来退出 Qemu。

ld rd, offset(rs1) x[rd] = M[x[rs1] + sext(offset)][63:0]双字加载 (Load Doubleword). I-type, RV32I and RV64I.从地址 x[rs1] + sign-extend(offset)读取八个字节，写入 x[rd]。压缩形式：c.ldsp rd, offset; c.ld rd, offset(rs1)offset[11:0] rs1 011 rd 0000011

mv rd, rs1 x[rd] = x[rs1]移动(Move). 伪指令(Pseudoinstruction), RV32I and RV64I.把寄存器 x[rs1]复制到 x[rd]中。实际被扩展为 addi rd, rs1, 0

保存寄存器和临时寄存器为什么不是连续编号的？为了支持 RV32E——一个只有 16 个寄存器的嵌入式版本的 RISC-V（参见第 11 章），只使用寄存器 x0 到 x15——一部分保存寄存器和一部分临时寄存器都在这个范围内。

保存返回地址（ra 寄存器）

根据 ABI 规范，我们来看看标准的 RV32I 函数入口和出口。下面是函数的开头：

jalr rd, offset(rs1) t =pc+4; pc=(x[rs1]+sext(offset))&~1; x[rd]=t跳转并寄存器链接 (Jump and Link Register). I-type, RV32I and RV64I.把 pc 设置为 x[rs1] + sign-extend(offset)，把计算出的地址的最低有效位设为 0，并将原 pc+4的值写入 f[rd]。rd 默认为 x1。压缩形式：c.jr rs1; c.jalr rs1offset[11:0] rs1 010 rd 110011

ret pc = x[1]返回(Return). 伪指令(Pseudoinstruction), RV32I and RV64I.从子过程返回。实际被扩展为 jalr x0, 0(x1)。

bne rs1, rs2, offset if (rs1 ≠ rs2) pc += sext(offset)不相等时分支 (Branch if Not Equal). B-type, RV32I and RV64I.若寄存器 x[rs1]和寄存器 x[rs2]的值不相等，把 pc 的值设为当前值加上符号位扩展的偏移offset。压缩形式：c.bnez rs1, offsetoffset[12|10:5] rs2 rs1 001 offset[4:1|11] 1100011

图 3.2 列出了寄存器的 RISC-V 应用程序二进制接口（ABI）名称和它们在函数调用中是否保留的规定。

RISC-V 有足够多的寄存器来达到两全其美的结果：既能将操作数存放在寄存器中，同时也能减少保存和恢复寄存器的次数。其中的关键在于，在函数调用的过程中不保留部分寄存器存储的值，称它们为临时寄存器；另一些寄存器则对应地称为保存寄存器。不再调用其它函数的函数称为叶函数。当一个叶函数只有少量的参数和局部变量时，它们可以都被存储在寄存器中，而不会“溢出（spilling）”到内存中。但如果函数参数和局部变量很多，程序还是需要把寄存器的值保存在内存中，不过这种情况并不多见。

第三章 RISC-V 汇编语言

图 2.4：RV32I 的寄存器

为常量 0 单独分配一个寄存器是 RISC-V ISA 能如此简单的一个很大的因素。第 3 章的第 36 页的图 3 给出了许多 ARM-32 和 x86-32 的原生指令操作，这两个指令集中没有零寄存器。我们可以用 RV32I 指令完成功能相同的操作，只需使用零寄存器作为操作数。

sd rs2, offset(rs1) M[x[rs1] + sext(offset) = x[rs2][63: 0]存双字(Store Doubleword). S-type, RV64I only.将 x[rs2]中的 8 字节存入内存地址 x[rs1]+sign-extend(offset)。压缩形式：c.sdsp rs2, offset; c.sd rs2, offset(rs1)offset[11:5] rs2 rs1 011 offset[4:0] 0100011

s0/fp Saved register/frame pointer

In RV64, 32-bit types, such as int, are stored in integer registers as proper sign extensions of their32-bit values; that is, bits 63..31 are all equal. This restriction holds even for unsigned 32-bit types.

Table 18.1 summarizes the datatypes natively supported by RISC-V C programs. In both RV32and RV64 C compilers, the C type int is 32 bits wide. longs and pointers, on the other hand, areboth as wide as a integer register, so in RV32, both are 32 bits wide, while in RV64, both are 64bits wide. Equivalently, RV32 employs an ILP32 integer model, while RV64 is LP64. In both RV32and RV64, the C type long long is a 64-bit integer, float is a 32-bit IEEE 754-2008 floating-pointnumber, double is a 64-bit IEEE 754-2008 floating-point number, and long double is a 128-bitIEEE floating-point number.The C types char and unsigned char are 8-bit unsigned integers and are zero-extended whenstored in a RISC-V integer register. unsigned short is a 16-bit unsigned integer and is zero-extended when stored in a RISC-V integer register. signed char is an 8-bit signed integer and issign-extended when stored in a RISC-V integer register, i.e. bits (XLEN-1)..7 are all equal. shortis a 16-bit signed integer and is sign-extended when stored in a register.

如果对某一个Stack Frame感兴趣，可以先定位到那个frame再输入info frame，假设对syscall的Stack Frame感兴趣。

如果输入backtrace（简写bt）可以看到从当前调用栈开始的所有Stack Frame。

如果我们在gdb中输入info frame，可以看到有关当前Stack Frame许多有用的信息。Stack level 0，表明这是调用栈的最底层pc，当前的程序计数器saved pc，demo4的位置，表明当前函数要返回的位置source language c，表明这是C代码Arglist at，表明参数的起始地址。当前的参数都在寄存器中，可以看到argc=3，argv是一个地址

学生提问，为什在最开始要对sp寄存器减16？TA：是为了Stack Frame创建空间。减16相当于内存地址向前移16，这样对于我们自己的Stack Frame就有了空间，我们可以在那个空间存数据。我们并不想覆盖原来在Stack Pointer位置的数据。学生提问：为什么不减4呢？TA：我认为我们不需要减16那么多，但是4个也太少了，你至少需要减8，因为接下来要存的ra寄存器是64bit（8字节）。这里的习惯是用16字节，因为我们要存Return address和指向上一个Stack Frame的地址，只不过我们这里没有存指向上一个Stack Frame的地址。如果你看kernel.asm，你可以发现16个字节通常就是编译器的给的值。

ra寄存器是0x80006392，它指向demo2函数，也就是sum_then_double的调用函数

leaf函数是指不调用别的函数的函数，它的特别之处在于它不用担心保存自己的Return address或者任何其他的Caller Saved寄存器，因为它不会调用别的函数。

在汇编代码中，函数的最开始你们可以看到Function prologue，之后是函数的本体，最后是Epilogue。这就是一个汇编函数通常的样子。

SP（Stack Pointer），它指向Stack的底部并代表了当前Stack Frame的位置。

FP（Frame Pointer），它指向当前Stack Frame的顶部

Return address总是会出现在Stack Frame的第一位

Stack Frame大小并不总是一样，即使在这个图里面看起来是一样大的。不同的函数有不同数量的本地变量，不同的寄存器，所以Stack Frame的大小是不一样的。

如果函数的参数多于8个，额外的参数会出现在Stack中。

For the C language, the asm keyword is a GNU extension. When writing C code that can be compiled with -ansi and the -std options that select C dialects without GNU extensions, use __asm__ instead of asm (see Alternate Keywords). For the C++ language, asm is a standard keyword, but __asm__ can be used for code compiled with -fno-asm.

an extended asm statement (see Extended Asm - Assembler Instructions with C Expression Operands) includes one or more operands. The extended form is preferred for mixing C and assembly language within a function, but to include assembly language at top level you must use basic asm.

QEMU有两种运行模式： User mode 模式，即用户态模拟，如 qemu-riscv64 程序， 能够模拟不同处理器的用户态指令的执行，并可以直接解析ELF可执行文件， 加载运行那些为不同处理器编译的用户级Linux应用程序。 System mode 模式，即系统态模式，如 qemu-system-riscv64 程序， 能够模拟一个完整的基于不同CPU的硬件系统，包括处理器、内存及其他外部设备，支持运行完整的操作系统。

#[panic_handler] 告知编译器采用我们的实现： // os/src/lang_items.rs use core::panic::PanicInfo; #[panic_handler] fn panic(_info: &PanicInfo) -> ! { loop {} }

在 main.rs 的开头加上一行 #![no_std]， 告诉 Rust 编译器不使用 Rust 标准库 std 转而使用核心库 core。

这种编译器运行的平台（x86_64）与可执行文件运行的目标平台不同的情况，称为 交叉编译 (Cross Compile)。

除了 std 之外，Rust 还有一个不需要任何操作系统支持的核心库 core， 它包含了 Rust 语言相当一部分核心机制，可以满足本门课程的需求。 有很多第三方库也不依赖标准库 std，而仅仅依赖核心库 core。

riscv64gc-unknown-none-elf 的 CPU 架构是 riscv64gc，厂商是 unknown，操作系统是 none， elf 表示没有标准的运行时库。没有任何系统调用的封装支持，但可以生成 ELF 格式的执行程序。 我们不选择有 linux-gnu 支持的 riscv64gc-unknown-linux-gnu，是因为我们的目标是开发操作系统内核，而非在 linux 系统上运行的应用程序。

目标三元组 (Target Triplet) 描述了目标平台的 CPU 指令集、操作系统类型和标准运行时库。

-w, --nodelist={<node_name_list>|<filename>}Request a specific list of hosts.

-N, --nodes=<minnodes>[-maxnodes]|<size_string>Request that a minimum of minnodes nodes be allocated to this job.

GresUsedGeneric resources (gres) currently in use on the nodes.

-t, --states=<states>List nodes only having the given state(s). Multiple states may be comma separated and the comparison is case insensitive. If the states are separated by '&', then the nodes must be in all states. Possible values include (case insensitive): ALLOC, ALLOCATED, CLOUD, COMP, COMPLETING, DOWN, DRAIN (for node in DRAINING or DRAINED states), DRAINED, DRAINING, FAIL, FUTURE, FUTR, IDLE, MAINT, MIX, MIXED, NO_RESPOND, NPC, PERFCTRS, PLANNED, POWER_DOWN, POWERING_DOWN, POWERED_DOWN, POWERING_UP, REBOOT_ISSUED, REBOOT_REQUESTED, RESV, RESERVED, UNK, and UNKNOWN. By default nodes in the specified state are reported whether they are responding or not. The --dead and --responding options may be used to filter nodes by the corresponding flag.

--jsonDump all node information as JSON. All information is dumped, even if it would normally not be. Formatting options are ignored; however, filtering arguments are still used.

The format of each field is "%[[.]size]type[suffix]" sizeMinimum field size. If no size is specified, whatever is needed to print the information will be used. .Indicates the output should be right justified and size must be specified. By default output is left justified. suffixArbitrary string to append to the end of the field.

-O, --Format=<output_format>Specify the information to be displayed. Also see the -o <output_format>, --format=<output_format> option (which supports greater flexibility in formatting, but does not support access to all fields because we ran out of letters)

之所以叫做二进制接口，是因为它和在同一种编程语言内部调用接口不同，是汇编指令级的一种接口。事实上 M/S/U 三个特权级的软件可能分别由不同的编程语言实现，即使是用同一种编程语言实现的，其调用也并不是普通的函数调用执行流，而是**陷入异常控制流** ，在该过程中会 切换 CPU 特权级。因此只有将接口下降到汇编指令级才能够满足其通用性和灵活性。

之前我们给出过支持应用程序运行的一套 执行环境栈 ，现在我们站在特权级架构的角度去重新看待它： 和之前一样，白色块表示一层执行环境，黑色块表示相邻两层执行环境之间的接口。这张图片给出了能够支持运行 Unix 这类复杂系统的软件栈。其中 内核代码运行在 S 模式上；应用程序运行在 U 模式上。运行在 M 模式上的软件被称为 监督模式执行环境 (SEE, Supervisor Execution Environment) ，这是站在运行在 S 模式上的软件的视角来看，它的下面也需要一层执行环境支撑，因此被命名为 SEE，它需要在相比 S 模式更高的特权级下运行， 一般情况下在 M 模式上运行。

RISC-V 架构中一共定义了 4 种特权级： RISC-V 特权级¶ 级别 编码 名称 0 00 用户/应用模式 (U, User/Application) 1 01 监督模式 (S, Supervisor) 2 10 H, Hypervisor 3 11 机器模式 (M, Machine) 其中，级别的数值越大，特权级越高，掌控硬件的能力越强。从表中可以看出， M 模式处在最高的特权级，而 U 模式处于最低的特权级。

如果测例仓库有所更新或者你切换了代码仓库的分支，你可能需要清理掉测例仓库原版的编译结果，此时需要执行 $ make -C user clean 它的作用基本等价于如下写法，但是更简便 $ cd user $ make clean $ cd ..

每次在make run之前，尽量先执行make clean以删除缓存，特别是在切换ch分支之后。

--meEquivalent to --user=<my username>.

The batch script may contain options preceded with "#SBATCH" before any executable commands in the script. sbatch will stop processing further #SBATCH directives once the first non-comment non-whitespace line has been reached in the script.

--export={[ALL,]<environment_variables>|ALL|NIL|NONE}Identify which environment variables from the submission environment are propagated to the launched application. Note that SLURM_* variables are always propagated. --export=ALLDefault mode if --export is not specified. All of the user's environment will be loaded (either from the caller's environment or from a clean environment if --get-user-env is specified).

filename pattern sbatch allows for a filename pattern to contain one or more replacement symbols, which are a percent sign "%" followed by a letter (e.g. %j).

Use the "B:" option to signal only the batch shell, none of the other processes will be signaled. By default all job steps will be signaled, but not the batch shell itself.

Use the "R:" option to allow this job to overlap with a reservation with MaxStartDelay set.

Due to the resolution of event handling by Slurm, the signal may be sent up to 60 seconds earlier than specified.

--signal=[{R|B}:]<sig_num>[@sig_time]When a job is within sig_time seconds of its end time, send it the signal sig_num.

--mem=<size>[units]Specify the real memory required per node.

use an enforcement mechanism. See ConstrainRAMSpace in the cgroup.conf(5) man page and OverMemoryKill in the slurm.conf(5) man page for more details.

Bootloader初始化外设，在存储设备上找到OS，加载执行OS

The weighted amount of a resource can be adjusted by adding a suffix of K,M,G,T or P after the billing weight. For example, a memory weight of "mem=.25" on a job allocated 8GB will be billed 2048 (8192MB *.25) units. A memory weight of "mem=.25G" on the same job will be billed 2 (8192MB * (.25/1024)) units.

You can launch projects from a repository on GitHub.com to your server by using a deploy key, which is an SSH key that grants access to a single repository.

The general form for this is “git-credential-foo [args] <action>.” The stdin/stdout protocol is the same as git-credential, but they use a slightly different set of actions: get is a request for a username/password pair. store is a request to save a set of credentials in this helper’s memory. erase purge the credentials for the given properties from this helper’s memory. For the store and erase actions, no response is required (Git ignores it anyway). For the get action, however, Git is very interested in what the helper has to say. If the helper doesn’t know anything useful, it can simply exit with no output, but if it does know, it should augment the provided information with the information it has stored. The output is treated like a series of assignment statements; anything provided will replace what Git already knows.

The credential system is actually invoking a program that’s separate from Git itself; which one and how depends on the credential.helper configuration value. There are several forms it can take: Configuration Value Behavior foo Runs git-credential-foo foo -a --opt=bcd Runs git-credential-foo -a --opt=bcd /absolute/path/foo -xyz Runs /absolute/path/foo -xyz !f() { echo "password=s3cre7"; }; f Code after ! evaluated in shell So the helpers described above are actually named git-credential-cache, git-credential-store, and so on, and we can configure them to take command-line arguments.

The osxkeychain and wincred helpers use the native format of their backing stores, while cache uses its own in-memory format (which no other process can read).

Git even allows you to configure several helpers. When looking for credentials for a particular host, Git will query them in order, and stop after the first answer is provided. When saving credentials, Git will send the username and password to all of the helpers in the list, and they can choose what to do with them.

The “store” helper can take a --file <path> argument, which customizes where the plain-text file is saved (the default is ~/.git-credentials).

Git has a few options provided in the box: The default is not to cache at all. Every connection will prompt you for your username and password. The “cache” mode keeps credentials in memory for a certain period of time. None of the passwords are ever stored on disk, and they are purged from the cache after 15 minutes. The “store” mode saves the credentials to a plain-text file on disk, and they never expire. This means that until you change your password for the Git host, you won’t ever have to type in your credentials again. The downside of this approach is that your passwords are stored in cleartext in a plain file in your home directory.

If you authenticate without GitHub CLI, you must authenticate with a personal access token. When Git prompts you for your password, enter your personal access token. Alternatively, you can use a credential helper like Git Credential Manager. Password-based authentication for Git has been removed in favor of more secure authentication methods.

If someone runs your script and you have to set anonymous="allow":Auto-create temporary account: W&B checks for an account that's already signed in. If there's no account, we automatically create a new anonymous account and save that API key for the session.Log results quickly: The user can run and re-run the script, and automatically see results show up in the W&B dashboard UI. These unclaimed anonymous runs will be available for 7 days.Claim data when it's useful: Once the user finds valuable results in W&B, they can easily click a button in the banner at the top of the page to save their run data to a real account. If they don't claim a run, it will be deleted after 7 days.

What happens to users with existing accounts?​If you set anonymous="allow" in your script, we will check to make sure there's not an existing account first, before creating an anonymous account. This means that if a W&B user finds your script and runs it, their results will be logged correctly to their account, just like a normal run.

There is currently no option to list multiple api keys in a single netrc file. If you were to do this manually, wandb will default to the last listed api key to authenticate the user.

The WandbCallback that Trainer uses will call wandb.init under the hood when Trainer is initialized. You can alternatively set up your runs manually by calling wandb.init before theTrainer is initialized. This gives you full control over your W&B run configuration.

Multiple wandb users on shared machines​If you're using a shared machine and another person is a wandb user, it's easy to make sure your runs are always logged to the proper account. Set the WANDB_API_KEY environment variable to authenticate. If you source it in your env, when you log in you'll have the right credentials, or you can set the environment variable from your script.Run this command export WANDB_API_KEY=X where X is your API key. When you're logged in, you can find your API key at wandb.ai/authorize.

WANDB_TAGSA comma separated list of tags to be applied to the run.

WANDB_RESUMEBy default this is set to never. If set to auto wandb will automatically resume failed runs. If set to must forces the run to exist on startup. If you want to always generate your own unique ids, set this to allow and always set WANDB_RUN_ID.

You can also turn on Source Mode globally, instead of on a document-by-document basis. This is helpful if you need to copy/paste a lot of data, or want to see what the data in your files looks like “behind the scenes”. To do this, open settings in Obsidian, and look under “Editor”. You should see an option that says “Properties in Document”. Change that to “Source”:

# Set the priority when loading # e.g., zsh-syntax-highlighting must be loaded # after executing compinit command and sourcing other plugins # (If the defer tag is given 2 or above, run after compinit command) zplug "zsh-users/zsh-syntax-highlighting", defer:2

在输出中野蛮地强制插入约束短语 is fast 的问题在于，大多数情况下，你最终会得到像上面的 The is fast 这样的无意义输出。我们需要解决这个问题。你可以从 huggingface/transformers 代码库中的这个 问题 中了解更多有关这个问题及其复杂性的深入讨论。 组方法通过在满足约束和产生合理输出两者之间取得平衡来解决这个问题。 我们把所有候选波束按照其 满足了多少步约束分到不同的组中，其中组 $n$ 里包含的是 满足了 $n$ 步约束的波束列表 。然后我们按照顺序轮流选择各组的候选波束。在上图中，我们先从组 2 (Bank 2) 中选择概率最大的输出，然后从组 1 (Bank 1) 中选择概率最大的输出，最后从组 0 (Bank 0) 中选择最大的输出; 接着我们从组 2 (Bank 2) 中选择概率次大的输出，从组 1 (Bank 1) 中选择概率次大的输出，依此类推。因为我们使用的是 num_beams=3，所以我们只需执行上述过程三次，就可以得到 ["The is fast", "The dog is", "The dog and"]。 这样，即使我们 强制 模型考虑我们手动添加的约束词分支，我们依然会跟踪其他可能更有意义的高概率序列。尽管 The is fast 完全满足约束，但这并不是一个有意义的短语。幸运的是，我们有 "The dog is" 和 "The dog and" 可以在未来的步骤中使用，希望在将来这会产生更有意义的输出。 图示如下 (以上例的第 3 步为例): 请注意，上图中不需要强制添加 "The is fast"，因为它已经被包含在概率排序中了。另外，请注意像 "The dog is slow" 或 "The dog is mad" 这样的波束实际上是属于组 0 (Bank 0) 的，为什么呢？因为尽管它包含词 "is" ，但它不可用于生成 "is fast" ，因为 fast 的位子已经被 slow 或 mad 占掉了，也就杜绝了后续能生成 "is fast" 的可能性。从另一个角度讲，因为 slow 这样的词的加入，该分支 满足约束的进度 被重置成了 0。

repetition_penalty 可用于对生成重复的单词这一行为进行惩罚。它首先由 Keskar 等人 (2019) 引入，在 Welleck 等人 (2019) 的工作中，它是训练目标的一部分。它可以非常有效地防止重复，但似乎对模型和用户场景非常敏感，其中一个例子见 Github 上的 讨论。

生成的文本看起来不错 - 但仔细观察会发现它不是很连贯。3-grams new hand sense 和 local batte harness 非常奇怪，看起来不像是人写的。这就是对单词序列进行采样时的大问题: 模型通常会产生不连贯的乱码，参见 Ari Holtzman 等人 (2019) 的论文。

在机器翻译或摘要等任务中，因为所需生成的长度或多或少都是可预测的，所以波束搜索效果比较好 - 参见 Murray 等人 (2018) 和 Yang 等人 (2018) 的工作。但开放域文本生成情况有所不同，其输出文本长度可能会有很大差异，如对话和故事生成的输出文本长度就有很大不同。

正如 Ari Holtzman 等人 (2019) 所论证的那样，高质量的人类语言并不遵循最大概率法则。换句话说，作为人类，我们希望生成的文本能让我们感到惊喜，而可预测的文本使人感觉无聊。论文作者画了一个概率图，很好地展示了这一点，从图中可以看出人类文本带来的惊喜度比波束搜索好不少。

波束搜索通过在每个时间步保留最可能的 num_beams 个词，并从中最终选择出概率最高的序列来降低丢失潜在的高概率序列的风险。以 num_beams=2 为例: 在时间步 1，除了最有可能的假设 $(\text{“The”}, \text{“nice”})$，波束搜索还跟踪第二可能的假设 $(\text{“The”}, \text{“dog”})$。在时间步 2，波束搜索发现序列 $(\text{“The”}, \text{“dog”}, \text{“has”})$ 概率为$0.36$，比 $(\text{“The”}, \text{“nice”}, \text{“woman”})$ 的 $0.2$ 更高。太棒了，在我们的例子中它已经找到了最有可能的序列！

虽然结果比贪心搜索更流畅，但输出中仍然包含重复。一个简单的补救措施是引入 n-grams (即连续 n 个词的词序列) 惩罚，该方法是由 Paulus 等人 (2017) 和 Klein 等人 (2017) 引入的。最常见的 n-grams 惩罚是确保每个 n-gram 都只出现一次，方法是如果看到当前候选词与其上文所组成的 n-gram 已经出现过了，就将该候选词的概率设置为 0。

模型很快开始输出重复的文本！这在语言生成中是一个非常普遍的问题，在贪心搜索和波束搜索中似乎更是如此 - 详见 Vijayakumar 等人，2016 和 Shao 等人，2017 的论文。

The only caveat seems to be that train loss doesn’t exist at this point so the wandb plots are offset. class EvaluateFirstStepCallback(TrainerCallback): def on_step_begin(self, args, state, control, **kwargs): if state.global_step == 1: control.should_evaluate = True trainer.add_callback(EvaluateFirstStepCallback())

auto_find_batch_size (bool, optional, defaults to False) — Whether to find a batch size that will fit into memory automatically through exponential decay, avoiding CUDA Out-of-Memory errors. Requires accelerate to be installed (pip install accelerate)

full_determinism (bool, optional, defaults to False) — If True, enable_full_determinism() is called instead of set_seed() to ensure reproducible results in distributed training. Important: this will negatively impact the performance, so only use it for debugging.

dataloader_num_workers (int, optional, defaults to 0) — Number of subprocesses to use for data loading (PyTorch only). 0 means that the data will be loaded in the main process.

debug (str or list of DebugOption, optional, defaults to "") — Enable one or more debug features. This is an experimental feature. Possible options are: "underflow_overflow": detects overflow in model’s input/outputs and reports the last frames that led to the event "tpu_metrics_debug": print debug metrics on TPU The options should be separated by whitespaces.

seed (int, optional, defaults to 42) — Random seed that will be set at the beginning of training. To ensure reproducibility across runs, use the ~Trainer.model_init function to instantiate the model if it has some randomly initialized parameters.

seed (int, optional, defaults to 42) — Random seed that will be set at the beginning of training. To ensure reproducibility across runs, use the ~Trainer.model_init function to instantiate the model if it has some randomly initialized parameters. data_seed (int, optional) — Random seed to be used with data samplers. If not set, random generators for data sampling will use the same seed as seed. This can be used to ensure reproducibility of data sampling, independent of the model seed.

logging_first_step (bool, optional, defaults to False) — Whether to log and evaluate the first global_step or not.

By default Trainer will use logging.INFO for the main process and logging.WARNING for the replicas if any. These defaults can be overridden to use any of the 5 logging levels with TrainingArguments’s arguments: log_level - for the main process log_level_replica - for the replicas

By default, Trainer will save all checkpoints in the output_dir you set in the TrainingArguments you are using. Those will go in subfolder named checkpoint-xxx with xxx being the step at which the training was at.

sharded_ddp (bool, str or list of ShardedDDPOption, optional, defaults to '') — Use Sharded DDP training from FairScale (in distributed training only). This is an experimental feature.

compute_metrics (Callable[[EvalPrediction], Dict], optional) — The function that will be used to compute metrics at evaluation. Must take a EvalPrediction and return a dictionary string to metric values.

preprocess_logits_for_metrics (Callable[[torch.Tensor, torch.Tensor], torch.Tensor], optional) — A function that preprocess the logits right before caching them at each evaluation step. Must take two tensors, the logits and the labels, and return the logits once processed as desired. The modifications made by this function will be reflected in the predictions received by compute_metrics.

remove_unused_columns (bool, optional, defaults to True) — Whether or not to automatically remove the columns unused by the model forward method. (Note that this behavior is not implemented for TFTrainer yet.)

($@) Expands to the positional parameters, starting from one. In contexts where word splitting is performed, this expands each positional parameter to a separate word; if not within double quotes, these words are subject to word splitting. In contexts where word splitting is not performed, this expands to a single word with each positional parameter separated by a space. When the expansion occurs within double quotes, and word splitting is performed, each parameter expands to a separate word. That is, "$@" is equivalent to "$1" "$2" …. If the double-quoted expansion occurs within a word, the expansion of the first parameter is joined with the beginning part of the original word, and the expansion of the last parameter is joined with the last part of the original word. When there are no positional parameters, "$@" and $@ expand to nothing (i.e., they are removed).

the modulecmd outputs valid shell commands to stdout which manipulates the shell's environment. Any text that is meant to be seen by the user must be sent to stderr. For example: puts stderr "\n\tSome Text to Show\n"

Path elements registered in the MODULEPATH environment variable may contain reference to environment variables which are converted to their corresponding value by module command each time it looks at the MODULEPATH value. If an environment variable referred in a path element is not defined, its reference is converted to an empty string. MODULERCFILE

MODULEPATH¶ The path that the module command searches when looking for modulefiles. Typically, it is set to the main modulefiles directory, /usr/share/Modules/modulefiles, by the initialization script.

Once the Modules package is initialized, the environment can be modified on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc.

操作系统结构

微内核结构(Micro Kernel) 尽可能把内核功能移到用户空间 用户模块间的通信使用消息传递 好处: 灵活/安全... 缺点: 性能

stingy QoS is for users who want to run jobs with the lowest priority without charging (UsageFactor=0). These jobs may be preempted (suspended, re-queued or terminated) when higher priority jobs are submitted to the system.

IncompleteOperate only on jobs (or tasks of a job array) which have not completed. Specifically only jobs in the following states will be requeued: CONFIGURING, RUNNING, STOPPED or SUSPENDED.

In your training program, you are supposed to call the following function at the beginning to start the distributed backend. It is strongly recommended that init_method=env://. Other init methods (e.g. tcp://) may work, but env:// is the one that is officially supported by this module. >>> torch.distributed.init_process_group(backend='YOUR BACKEND', >>> init_method='env://')

torch.distributed.barrier(group=None, async_op=False, device_ids=None)[source] Synchronizes all processes. This collective blocks processes until the whole group enters this function, if async_op is False, or if async work handle is called on wait().

可以使用其他的device map映射方式，通过设置device_map参数（例如"auto", "balanced", "balanced_low_0", "sequential"），或者手工设置这个字典（如果控制欲很强，或者有特殊需求）。读者可以操控模型在meta设备上的所有层（计算device_map）。当读者没有足够GPU显存来加载完整的模型（因为都会按照先占满GPU显存，再占满CPU/内存资源，最后占据硬盘的顺序来完成模型加载），上面所有的选项得到的层和设备对应结果将会相同。当读者有足够的GPU资源来加载模型，那么上面4个选项得到的结果会有所不同。"auto" 和 "balanced" 将会在所有的GPU上平衡切分模型，那么可以计算批尺寸大于1的输入"balanced_low_0" 会在除了第一个GPU上的其它GPU上平衡划分模型，并且在第一个GPU上占据较少资源。这个选项符合需要在第一个GPU上进行额外操作的需求，例如需要在第一个GPU执行generate函数（迭代过程）。"sequential" 按照GPU的顺序分配模型分片，从GPU 0开始，直到最后的GPU（那么最后的GPU往往不会被占满，和"balanced_low_0"的区别就是第一个还是最后一个，以及非均衡填充）这里"auto"和"balanced" 会得到相同的结果，但是未来"auto"模式可能会被改变，主要是有可能发现更高效的分配策略。"balanced"参数的功能则保持稳定。

A maximum number of simultaneously running tasks from the job array may be specified using a "%" separator. For example "--array=0-15%4" will limit the number of simultaneously running tasks from this job array to 4.

出现问题的条件 在 pytorch 1.5 + 以上的版本 在多卡训练 在import torch 在 import numpy 之前 原因 如果在 numpy 之前导入了 torch，那么这里的子进程将获得一个 GNU 线程层（即使父进程没有定义变量） 但是如果 numpy 在 Torch 之前被导入，子进程将获得一个 INTEL 线程层，这种情况会导致线程之间打架

OS 原理与设计思想 操作系统结构 中断及系统调用 内存管理 进程管理 处理机调度 同步互斥 文件系统 I/O 子系统

wandb login [OPTIONS] [KEY]...

fallback selects an available policy by priority. The availability is tested by accessing an URL, just like an auto url-test group.

you can use RESTful API to switch proxy is recommended for use in GUI.

If you prefer to launch your training job using MPI (e.g., mpirun), we provide support for this. It should be noted that DeepSpeed will still use the torch distributed NCCL backend and not the MPI backend.

If you would like to propagate additional variables you can specify them in a dot-file named .deepspeed_env that contains a new-line separated list of VAR=VAL entries. The DeepSpeed launcher will look in the local path you are executing from and also in your home directory (~/). If you would like to override the default name of this file or path and name with your own, you can specify this with the environment variable, DS_ENV_FILE.

By default DeepSpeed will propagate all NCCL and PYTHON related environment variables that are set.

Important: all processes must call this method and not just the process with rank 0. It is because each process needs to save its master weights and scheduler+optimizer states. This method will hang waiting to synchronize with other processes if it’s called just for the process with rank 0.

the user may want to save additional data that are unique to a given model training. To support these items, save_checkpoint accepts a client state dictionary client_sd for saving. These items can be retrieved from load_checkpoint as a return argument.

Learning Rate Scheduler: when using a DeepSpeed’s learning rate scheduler (specified in the ds_config.json file), DeepSpeed calls the step() method of the scheduler at every training step (when model_engine.step() is executed). When not using DeepSpeed’s learning rate scheduler: if the schedule is supposed to execute at every training step, then the user can pass the scheduler to deepspeed.initialize when initializing the DeepSpeed engine and let DeepSpeed manage it for update or save/restore. if the schedule is supposed to execute at any other interval (e.g., training epochs), then the user should NOT pass the scheduler to DeepSpeed during initialization and must manage it explicitly.

Once the DeepSpeed engine has been initialized, it can be used to train the model using three simple APIs for forward propagation (callable object), backward propagation (backward), and weight updates (step). for step, batch in enumerate(data_loader): #forward() method loss = model_engine(batch) #runs backpropagation model_engine.backward(loss) #weight update model_engine.step()

To initialize the DeepSpeed engine: model_engine, optimizer, _, _ = deepspeed.initialize(args=cmd_args, model=model, model_parameters=params) deepspeed.initialize ensures that all of the necessary setup required for distributed data parallel or mixed precision training are done appropriately under the hood. In addition to wrapping the model, DeepSpeed can construct and manage the training optimizer, data loader, and the learning rate scheduler based on the parameters passed to deepspeed.initialize and the DeepSpeed configuration file.

If you already have a distributed environment setup, you’d need to replace: torch.distributed.init_process_group(...) with: deepspeed.init_distributed()

But if you don’t need the distributed environment setup until after deepspeed.initialize() you don’t have to use this function, as DeepSpeed will automatically initialize the distributed environment during its initialize.

The default is to use the NCCL backend, which DeepSpeed has been thoroughly tested with, but you can also override the default.

DeepSpeed on AMD can be used via our ROCm images, e.g., docker pull deepspeed/rocm501:ds060_pytorch110.

If no hostfile is specified, DeepSpeed searches for /job/hostfile. If no hostfile is specified or found, DeepSpeed queries the number of GPUs on the local machine to discover the number of local slots available.

A hostfile is a list of hostnames (or SSH aliases), which are machines accessible via passwordless SSH, and slot counts, which specify the number of GPUs available on the system. For example, worker-1 slots=4 worker-2 slots=4 specifies that two machines named worker-1 and worker-2 each have four GPUs to use for training.

By default DeepSpeed expects that a multi-node environment uses a shared storage. If this is not the case and each node can only see the local filesystem, you need to adjust the config file to include a checkpoint_section with the following setting: Copied { "checkpoint": { "use_node_local_storage": true } } Alternatively, you can also use the Trainer’s --save_on_each_node argument, and the above config will be added automatically for you.

For example, to use torch.distributed.run, you could do: Copied python -m torch.distributed.run --nproc_per_node=8 --nnode=2 --node_rank=0 --master_addr=hostname1 \ --master_port=9901 your_program.py <normal cl args> --deepspeed ds_config.json You have to ssh to each node and run this same command on each one of them! There is no rush, the launcher will wait until both nodes will synchronize.

The information in this section isn’t not specific to the DeepSpeed integration and is applicable to any multi-node program. But DeepSpeed provides a deepspeed launcher that is easier to use than other launchers unless you are in a SLURM environment.

Stage 0 is disabling all types of sharding and just using DeepSpeed as DDP. You can turn it on with: Copied { "zero_optimization": { "stage": 0 } } This will essentially disable ZeRO without you needing to change anything else.

MATCH,policy routes the rest of the packets to policy. This rule is required and is usually used as the last rule.

curl -X PUT -H "Content-Type: application/json" -d '{"name":"节点名"}' http://localhost:port/proxies/:Selector -H 添加 HTTP 请求的标头Content-Type: application/json，根据链接2，不设置标头为application/json可能会有问题。 -d 参数用于发送 POST 请求的数据体。 最后的网址为clash的external-controller的网址端口，最后Selector为要选择的proxy-groups的名称。 实际指令类似下面这条： 1curl -X PUT -H "Content-Type: application/json" -d '{"name":"HongKong"}' http://127.0.0.1:9090/proxies/Proxy

PUT 切换 Selector 中选中的代理/proxies/:name （这边的:name可以为节点名称，也可以为Selector。只要在proxies/后直接加上字符串就可以，不需要引号或者:）

External Controller enables users to control Clash programmatically with the HTTP RESTful API. The third-party Clash GUIs are heavily based on this feature. Enable this feature by specifying an address in external-controller.

看一看 youtube.com 这条规则加在那个分组里 - DOMAIN-SUFFIX,youtube.com,🌍 国外媒体可以看到 youtube 是在 国外媒体分组里，那么在这里要看国外媒体选择的是那个节点可以看到国外媒体选择的是 香港2-5 这个节点，选择是的延迟最低的节点，所以 youtube 走的是这个代理。

While Clash is meant to be run in the background, there's currently no elegant way to implement daemons with Golang, hence we recommend you to daemonize Clash with third-party tools.

If you want to place your configurations elsewhere (e.g. /etc/clash), you can use command-line option -d to specify a configuration directory:shellclash -d . # current directory clash -d /etc/clashOr, you can use option -f to specify a configuration file:shellclash -f ./config.yaml clash -f /etc/clash/config.yaml

The main configuration file is called config.yaml. By default, Clash reads the configuration files at $HOME/.config/clash.

ShadowsocksR（简称SSR）是一种基于Socks5代理方式的加密传输协议，也可以指实现这个协议的各种开发包。

脂溢性皮炎患者皮肤上都有大量的马拉色菌定植，它们能释放出溶脂酶，而溶质酶可以将皮脂中的甘油三酯分解成油酸和花生四烯酸等游离脂肪酸，这些代谢物会导致角质细胞的分化异常，影响表皮和角质层的正常代谢功能，造成屏障功能受损和角化不全。其中，角化不全会导致角栓形成，也就是堆积的角质+皮脂+微生物及其代谢产物形成的毛囊栓塞，即黑头的基础；屏障功能受损会导致更多的炎症，炎症反应又会进一步影响角质分化，然后恶性循环。。。为了帮助大家理解，我简化一下这个逻辑链吧：马拉色菌（某品种真菌）——产生溶脂酶——皮脂被分解成油酸、花生四烯酸等游离脂肪酸——角质细胞分化异常——角栓形成/屏障功能受损——黑头/炎症——炎症进一步影响角质分化——加剧角栓——进入恶性循环~而甲硝唑是明确有效的抗真菌药，可以干掉定植在人皮肤表面的马拉色菌，自然后面描述的这串恶性循环也就无法开启了~“黑头宝宝”也就失去了赖以生存的肥沃土壤了。事实上，在大量翻阅资料的过程中我发现，并不仅是马拉色菌有这个分解甘油三酯的能力，痘肌宝宝都听过的痤疮丙酸杆菌也可以

皮肤相关的主要作用：抗厌氧菌（比如痤疮丙酸杆菌）+抗真菌（比如马拉色菌）+抗毛囊虫（即蠕形螨，俗称的螨虫）+免疫调节+抗炎抗氧化；作用机理：抗厌氧菌——甲硝唑的硝基在无氧环境中可以还原成氨基（等中间产物），并作用于厌氧菌的DNA，使其链断裂，或阻断其转录、复制等过程，从而杀灭厌氧菌；绝大多数专性厌氧菌能被8~16ug/g或更低浓度的甲硝唑所杀灭（这数据感觉屌屌的）；在厌氧菌细胞内被铁硫蛋白还原，其还原产物可抑制细菌DNA的合成；抗真菌——直接损伤真菌的胞质膜（比如核膜、线粒体膜、溶酶体膜等），使其通透性改变，造成细胞内重要物质的摄取障碍或漏失而致真菌死亡；抗毛囊虫——甲硝唑的硝基在无氧环境中可以还原成氨基（等中间产物），与蠕形螨虫体细胞分子发生反应而导致蠕形螨死亡；免疫调节——甲硝唑可以通过其选择素与综合素作用干扰免疫系统，包括干扰白细胞和内皮细胞的粘附作用、抑制炎症细胞的移行等，从而达到免疫调节功能；抗炎抗氧化——甲硝唑可灭活、减少炎症中产生的ROS，达到局部抗炎的效果。

黑色素说则要比污垢说晚一百年，主要观点是认为你看到的黑头表面的黑色主要来源是由角质细胞带上来的黑色素。对，就是你们深恶痛绝的黑素细胞产生的那种黑色素！（当然，也可能同时夹杂有皮脂及其衍生物、细菌及其代谢物、甚至不小心和它们吸附在一起的极少量污垢颗粒之类的，只是这些东西并非主要因素）不过，关于黑色素说其实也有争论，在这个假说出现的几年后有人报道说通过对黑头的超微结构观察后发现，并没有看到黑头里有黑色素颗粒的存在，而只看到了越靠近黑头表面（也就是最黑的那部分），角质细胞的堆积越致密且高度定向，所以，他们猜测黑头表面的黑色更可能是这种高度定向的角质细胞致密堆积后造成的

The moderations endpoint is a tool you can use to check whether content complies with OpenAI's usage policies. Developers can thus identify content that our usage policies prohibits and take action, for instance by filtering it.The models classifies the following categories:

The frequency and presence penalties found in the Chat completions API and Legacy Completions API can be used to reduce the likelihood of sampling repetitive sequences of tokens. They work by directly modifying the logits (un-normalized log-probabilities) with an additive contribution.mu[j] -> mu[j] - c[j] * alpha_frequency - float(c[j] > 0) * alpha_presence

For reference, gpt-3.5-turbo performs at a similar capability level to text-davinci-003 but at 10% the price per token!

The response format is similar to the response format of the Chat completions API but also includes the optional field logprobs.

For example, if you want to use secondary GPU, put "1".(add a new line to webui-user.bat not in COMMANDLINE_ARGS): set CUDA_VISIBLE_DEVICES=0Alternatively, just use --device-id flag in COMMANDLINE_ARGS.

If you want to force the model to call a specific function you can do so by setting function_call: {"name": "<insert-function-name>"}. You can also force the model to generate a user-facing message by setting function_call: "none". Note that the default behavior (function_call: "auto") is for the model to decide on its own whether to call a function and if so which function to call.

Hallucinated outputs in function calls can often be mitigated with a system message. For example, if you find that a model is generating function calls with functions that weren't provided to it, try using a system message that says: "Only use the functions you have been provided with."

You can see these steps in action through the example below:

In the process of training a neural network, there are multiple stages where randomness is used, for example random initialization of weights of the network before the training starts. regularization, e.g. dropout, which involves randomly dropping nodes in the network while training. optimization process like stochastic gradient descent, RMSProp or Adam also include random initializations.

Under the hood, functions are injected into the system message in a syntax the model has been trained on. This means functions count against the model's context limit and are billed as input tokens.

The latest models (gpt-3.5-turbo-0613 and gpt-4-0613) have been fine-tuned to both detect when a function should to be called (depending on the input) and to respond with JSON that adheres to the function signature.

For most models, this is 4,096 tokens or about 3,000 words. As a rough rule of thumb, 1 token is approximately 4 characters or 0.75 words for English text.

The completions endpoint is the core of our API and provides a simple interface that’s extremely flexible and powerful.

As of PyTorch v1.6.0, features in torch.distributed can be categorized into three main components:

We're using the time since last checkpoint to determine if we should checkpoint this - this will very slightly vary between the different processes. Eventually, there comes a batch during which one of these two processes enters the intra-epoch checkpoint saving block and the other does not. In the intra-epoch saving block, the run_on_main function has a ddp.barrier synchronisation routine. I think in this case we don't need synchronisation, we just need to run the saving on the main process exclusively. If that is so, the fix would luckily be simple.

In this case, rank 6 is performing a different collective than other ranks. It’s performing a barrier while the others are performing a broadcast. Could it be that you have an uneven dataset and rank 6 is done while the other still going? If this is the case, you can try to use the join context manager 48.

The most common issues with collectives are shape and ordering mismatch. This is why Pritam’s suggestion of printing the input shapes could help you understand which issue you’re facing here.

To enable RoPE scaling, simply pass --rope-scaling, --max-input-length and --rope-factors flags when running through CLI. --rope-scaling can take the values linear or dynamic. If your model is not fine-tuned to a longer sequence length, use dynamic. --rope-factor is the ratio between the intended max sequence length and the model’s original max sequence length.

The answer is yes, but you need two things as discussed in the issue mentioned above: use find_unused_parameters=False make sure that all model parameters contribute to the calculation of the loss. Here's an example

You need to have dpp_find_unused_parameters=False when using gradient_checkpointing=True.

no_sync()[source] A context manager to disable gradient synchronizations across DDP processes. Within this context, gradients will be accumulated on module variables, which will later be synchronized in the first forward-backward pass exiting the context.

In order to support this arbitrary inclusion/exclusion our launcher sets the appropriate CUDA_VISIBLE_DEVICES at process launch time on each node. This means that if the user sets their own CUDA_VISIBLE_DEVICES on the launching node it's not clear if they want to set this value on the local node or all nodes.

If you do not provide the deepspeed launcher with a hostfile (via -H/--hostfile) it will only launch processes within that local node.

deepspeed.zero.GatheredParameters是DeepSpeed提供的一个上下文管理器， # 它可以将分布在多个设备上的参数收集到一起。这部分参数保存在CPU上。

DeepSpeed can automatically detect the following external parameter scenarios:

he tensor embeddings.weight is used in both embeddings.forward() and compute_logits(). We call embeddings.weight an external parameter because it is used in the training loop outside of its owning module’s forward pass.

However, in some cases a parameter may be used outside of its module’s forward pass. We call these external parameters. ZeRO-3 can coordinate these parameters if they are registered either automatically or manually.

Some models partitioned with deepspeed.zero.Init may need to access a module’s weights outside of the class constructor or its forward() method. We refer to these weights as external parameters, since these parameters are accessed outside of the module that created them.

enable_input_require_grads < source > ( ) Enables the gradients for the input embeddings. This is useful for fine-tuning adapter weights while keeping the model weights fixed.

clean_up_tokenization_spaces (bool, optional, defaults to True) — Whether or not the model should cleanup the spaces that were added when splitting the input text during the tokenization process.