日志(log)
为了让自己的思路更加清晰, 下面我都会称日志为 log. 因为日志这个词有两种含义, 详情见百度百科释义或者维基百科释义 https://zh.wikipedia.org/wiki/日志 .
日记 https://zh.wikipedia.org/wiki/日记 的另一种说法."志" 字本身为 "记录" 的意思, 日志就为每日的记录(通常是跟作者有关的).
服务器日志(server log), 记录服务器等电脑设备或软件的运作.
我们这里说的当然是服务器日志, 也就是 server log .
写入 log
一般写入 log 都会遵循以下步骤:
- int fd = open(path)
- write(fd, sign_append)
- fclose(fd)
解释一下上面的代码:
1. int fd = open(path)
会通过系统调用打开一个文件描述符, 或者在其他语言中也可以称作资源描述符, 资源类型, 或句柄.
2. write(fd, append = 1)
write 系统调用, 并加上 append 标志, 会执行 seek 和 write 两个系统调用, 但是这种系统调用是原子性的.
原子性意味着 seek 和 write 会同时执行, 不会有两个线程产生交叉, 必须 a 线程执行完 seek 和 write ,b 线程才能继续执行(这里说线程, 是因为线程才是 CPU 调度的基本单位).
所以在 nginx 中, 我们加上 append 标志, 就不用对线程上锁了.
3. fclose(fd)
关闭描述符.
Linux 一般对打开的文件描述符有一个最大数量的限制, 如果不关闭描述符, 很有可能造成大 bug.
查看 Linux 中限制的方法如下(其中 open files 代表可以打开的文件数量):
- $ ulimit -a
- core file size (blocks, -c) 0
- data seg size (kbytes, -d) unlimited
- scheduling priority (-e) 0
- file size (blocks, -f) unlimited
- pending signals (-i) 15732
- max locked memory (kbytes, -l) 64
- max memory size (kbytes, -m) unlimited
- open files (-n) 1024
- pipe size (512 bytes, -p) 8
- POSIX message queues (bytes, -q) 819200
- real-time priority (-r) 0
- stack size (kbytes, -s) 8192
- CPU time (seconds, -t) unlimited
- max user processes (-u) 15732
- virtual memory (kbytes, -v) unlimited
- file locks (-x) unlimited
所以, 如果是系统调用, 那么 append 不用加锁.
为什么 PHP 语言写日志时用了 append 也要加锁?
如果根据上面的说法, 咱们可以设置好 write 的 append 标志, 然后就可以睡大觉去了, 文件永远不会冲突.
但是 (一般都有个但是) 你去看 PHP 的框架中都会在 file_put_contents 的 append 之前加锁.
于是, 怀疑是因为 file_put_contents 的底层实现没有实现原子性.
跟进源码(非 PHP 程序员或者对 PHP 底层源码无兴趣的可以跳过了):
file_put_contents 底层实现:
- // file.c
- /* {{{ proto int|false file_put_contents(string file, mixed data [, int flags [, resource context]])
- Write/Create a file with contents data and return the number of bytes written */
- PHP_FUNCTION(file_put_contents)
- {
- ...
- case IS_STRING:
- if (Z_STRLEN_P(data)) {
- numbytes = php_stream_write(stream, Z_STRVAL_P(data), Z_STRLEN_P(data));
- if (numbytes != Z_STRLEN_P(data)) {
- php_error_docref(NULL, E_WARNING, "Only %zd of %zd bytes written, possibly out of free disk space", numbytes, Z_STRLEN_P(data));
- numbytes = -1;
- }
- }
- break;
- ...
- }
- // php_streams.h
- PHPAPI ssize_t _php_stream_write(php_stream *stream, const char *buf, size_t count);
- #define php_stream_write_string(stream, str) _php_stream_write(stream, str, strlen(str))
- #define php_stream_write(stream, buf, count) _php_stream_write(stream, (buf), (count))
- // streams.c
- PHPAPI ssize_t _php_stream_write(php_stream *stream, const char *buf, size_t count)
- {
- ...
- if (stream->writefilters.head) {
- bytes = _php_stream_write_filtered(stream, buf, count, PSFS_FLAG_NORMAL);
- } else {
- bytes = _php_stream_write_buffer(stream, buf, count);
- }
- if (bytes) {
- stream->flags |= PHP_STREAM_FLAG_WAS_WRITTEN;
- }
- return bytes;
- }
- /* Writes a buffer directly to a stream, using multiple of the chunk size */
- static ssize_t _php_stream_write_buffer(php_stream *stream, const char *buf, size_t count){
- ...
- while (count> 0) {
- ssize_t justwrote = stream->ops->write(stream, buf, count);
- if (justwrote <= 0) {
- /* If we already successfully wrote some bytes and a write error occurred
- * later, report the successfully written bytes. */
- if (didwrite == 0) {
- return justwrote;
- }
- return didwrite;
- }
- buf += justwrote;
- count -= justwrote;
- didwrite += justwrote;
- /* Only screw with the buffer if we can seek, otherwise we lose data
- * buffered from fifos and sockets */
- if (stream->ops->seek && (stream->flags & PHP_STREAM_FLAG_NO_SEEK) == 0) {
- stream->position += justwrote;
- }
- }
- }
- // php_streams.h
- /* operations on streams that are file-handles */
- typedef struct _php_stream_ops {
- /* stdio like functions - these are mandatory! */
- ssize_t (*write)(php_stream *stream, const char *buf, size_t count);
- ssize_t (*read)(php_stream *stream, char *buf, size_t count);
- int (*close)(php_stream *stream, int close_handle);
- int (*flush)(php_stream *stream);
- const char *label; /* label for this ops structure */
- /* these are optional */
- int (*seek)(php_stream *stream, zend_off_t offset, int whence, zend_off_t *newoffset);
- int (*cast)(php_stream *stream, int castas, void **ret);
- int (*stat)(php_stream *stream, php_stream_statbuf *ssb);
- int (*set_option)(php_stream *stream, int option, int value, void *ptrparam);
- } php_stream_ops;
- // plain_wrapper.c
- static ssize_t php_stdiop_write(php_stream *stream, const char *buf, size_t count)
- {
- php_stdio_stream_data *data = (php_stdio_stream_data*)stream->abstract;
- assert(data != NULL);
- if (data->fd>= 0) {
- #ifdef PHP_WIN32
- ssize_t bytes_written;
- if (ZEND_SIZE_T_UINT_OVFL(count)) {
- count = UINT_MAX;
- }
- bytes_written = _write(data->fd, buf, (unsigned int)count);
- #else
- ssize_t bytes_written = write(data->fd, buf, count);
- #endif
- if (bytes_written <0) {
- if (errno == EWOULDBLOCK || errno == EAGAIN) {
- return 0;
- }
- if (errno == EINTR) {
- /* TODO: Should this be treated as a proper error or not? */
- return bytes_written;
- }
- php_error_docref(NULL, E_NOTICE, "write of %zu bytes failed with errno=%d %s", count, errno, strerror(errno));
- }
- return bytes_written;
- } else {
- #if HAVE_FLUSHIO
- if (data->is_seekable && data->last_op == 'r') {
- zend_fseek(data->file, 0, SEEK_CUR);
- }
- data->last_op = 'w';
- #endif
- return (ssize_t) fwrite(buf, 1, count, data->file);
- }
- }
这个函数最终调用的是函数 php_stdiop_write
函数 _php_stream_write_buffer 中会将字符串分成多个 chunksize , 每个 chunksize 为 8192 (8K) 字节, 分别进行 write.
如果不加锁, 那么超过 8192 字节之后, 多个进程写日志就会出现混乱.
而且, PHP 文档也说明了:
所以, 最终需要根据不同的语言, 具体分析.
来源: https://www.cnblogs.com/wudanyang/p/12113348.html