Ben Gras [Mon, 16 Sep 2013 20:52:36 +0000 (22:52 +0200)]
netbsd dirent.h, import sys/sys *.h, mfs cleanup
. add all sys/sys headers not already present to help compiling
. take netbsd dirent.h and struct dirent; main result is
introducing d_type and d_namlen that have to be set by getdents()
in all FS code implementing it
. d_off is gone
. alignment of the struct has become 8 bytes instead of 4
. remove _MAX_BLOCK_SIZE, _MIN_BLOCK_SIZE, _STATIC_BLOCK_SIZE
. libminlib: cleanup unused yet duplicate code
. mfs: throw out the long-broken v1, v2 support
. new test for dirent contents filled by getdents()
- all TTY-related exceptions have now been merged into the regular
code paths, allowing non-TTY drivers to expose TTY-like devices;
- as part of this, CTTY_MAJOR is now fully managed by VFS instead of
being an ugly stepchild of the TTY driver;
- device styles have become completely obsolete, support for them has
been removed throughout the system; same for device flags, which had
already become useless a while ago;
- device map open/close and I/O function pointers have lost their use,
thus finally making the VFS device code actually readable;
- the device-unrelated pm_setsid has been moved to misc.c;
- some other small cleanup-related changes.
This single function allows copying file descriptors from and to
processes, and closing a previously copied remote file descriptor.
This function replaces the five FD-related UDS backcalls. While it
limits the total number of in-flight file descriptors to OPEN_MAX,
this change greatly improves crash recovery support of UDS, since all
in-flight file descriptors will be closed instead of keeping them
open indefinitely (causing VFS to crash on system shutdown). With the
new copyfd call, UDS becomes simpler, and the concept of filps is no
longer exposed outside of VFS.
This patch also moves the checkperms(2) stub into libminlib, thus
fully abstracting away message details of VFS communication from UDS.
- sendmsg: the accumulation of multiple in-flight file descriptors was
already described in the comments; now the code actually does what
the comments say :) -- also, added robustness in case of a failure;
- recvmsg: only create a socket rights message if there are file
descriptors pending at all;
- recvmsg: copy back the control message length;
- recvmsg: use CMSG_SPACE instead of CMSG_LEN to compute sizes.
Not sure if all of this is now working according to specification,
but at least tmux seems to be happy with it.
Well, make a start, anyway. Our copy was missing a legacy field from
the structure, that could very well cause applications to fail trying
to set, clear, or check it. As a consequence, SUN_LEN now yields the
same result as on NetBSD.
- move VFS calls to a separate source file;
- solve a few subtle bugs, mostly in error handling;
- simplify debug reporting code;
- make a few definitions more independent;
- restyle to something closer to KNF.
at_wini: PCI-only now; one controller per instance
- remove non-PCI support, since all supported platforms with at_wini
devices also have PCI support by now;
- correspondingly, stop using information from the BIOS altogether;
- limit each driver instance to one controller, to be in line with
the general MINIX3 one-instance-per-controller driver model; this
limits the number of disks per at_wini instance to four;
- go through the controllers by the order of their occurrence in the
PCI table, thus removing the exception for compatibility devices;
- let the second at_wini instance shut down silently if there is only
one IDE controller;
- clean up some extra code we don't need anymore, and resolve some
WARNS=5 level warnings.
Overall, these changes should simplify automatic loading of the right
disk drivers at boot time in the future.
- change "vid/did" to "vid:did", old form still supported for now;
- allow "vid:did/subvid:subdid" specification in system.conf, in
which case a device will be visible to a driver if the subsystem
VID/DID also match.
Lionel Sambuc [Fri, 4 Oct 2013 13:41:18 +0000 (15:41 +0200)]
Remove obsolete compatibility links
* /usr/bin/strip would be replaced by a link, even when compiling
binutils.
* Updated minix-spec.h to use the correct dynamic linker, and removed
the previously used symlinks.
* Removed the patch to the builtin PATH of ld.elf_so.
Lionel Sambuc [Thu, 3 Oct 2013 20:39:47 +0000 (22:39 +0200)]
Splitting commands/cd/cd.sh
As the situation is right now, importing one of the commands the
script replaces, requires a doc/UPDATING step.
By moving the script to a shared folder, and symlinking it once per
command, this allows for separatly installed files on the system,
instead of one file being symlinked multiple times.
This commit separates the low-level keyboard driver from TTY, putting
it in a separate driver (PCKBD). The commit also separates management
of raw input devices from TTY, and puts it in a separate server
(INPUT). All keyboard and mouse input from hardware is sent by drivers
to the INPUT server, which either sends it to a process that has
opened a raw input device, or otherwise forwards it to TTY for
standard processing.
Design by Dirk Vogt. Prototype by Uli Kastlunger.
Additional changes made to the prototype:
- the event communication is now based on USB HID codes; all input
drivers have to use USB codes to describe events;
- all TTY keymaps have been converted to USB format, with the effect
that a single keymap covers all keys; there is no (static) escaped
keymap anymore;
- further keymap tweaks now allow remapping of literally all keys;
- input device renumbering and protocol rewrite;
- INPUT server rewrite, with added support for cancel and select;
- PCKBD reimplementation, including PC/AT-to-USB translation;
- support for manipulating keyboard LEDs has been added;
- keyboard and mouse multiplexer devices have been added to INPUT,
primarily so that an X server need only open two devices;
- a new "libinputdriver" library abstracts away protocol details from
input drivers, and should be used by all future input drivers;
- both INPUT and PCKBD can be restarted;
- TTY is now scheduled by KERNEL, so that it won't be punished for
running a lot; without this, simply running "yes" on the console
kills the system;
- the KIOCBELL IOCTL has been moved to /dev/console;
- support for the SCANCODES termios setting has been removed;
- obsolete keymap compression has been removed;
- the obsolete Olivetti M24 keymap has been removed.
Due to the existence of /dev/console and /dev/log, and the new
"console=" setting, it is now possible that a single non-PTY object
(e.g. serial) is accessible through two different minor numbers. This
poses a problem when sending late select replies (CDEV_SEL2_REPLY),
because the object's minor number can not be used to identify the
device. Since selecting on such objects through translated minor
numbers is actually required, we now save the minor number used to
initiate the select query in order to send a late reply.
The solution is suboptimal, as it is not possible to use two different
minors to select on the same object at once. In the future, there
should be at least one select record for each minor that can be used
with each object.
Thomas Cort [Sat, 7 Sep 2013 01:40:31 +0000 (21:40 -0400)]
uname: normalize release and version
Most systems provide the full version number in the
'release' field and the kernel version in 'version'.
Minix used to split the full version number between
release and version which caused problems for pkgsrc
and other applications. This patch brings Minix's
uname in line with other systems such as NetBSD.
It also brings the getty banner in line with NetBSD.
Old Minix uname:
sysname->Minix
nodename->10.0.2.15
release->3
version->2.1
machine->i686
New Minix uname:
sysname->Minix
nodename->10.0.2.15
release->3.2.1
version->Minix 3.2.1 (GENERIC)
machine->i686
The set of processes to which a SIGKMESS signal is sent whenever new
diagnostics messages are added to the kernel's message buffer, is now
no longer hardcoded. Instead, processes can (un)register themselves
to receive such notifications, by means of sys_diagctl().
* Renamed struct timer to struct minix_timer
* Renamed timer_t to minix_timer_t
* Ensured all the code uses the minix_timer_t typedef
* Removed ifdef around _BSD_TIMER_T
* Removed include/timers.h and merged it into include/minix/timers.h
* Resolved prototype conflict by renaming kernel's (re)set_timer
to (re)set_kernel_timer.
This call copies a file descriptor from a remote process into the
calling process. The call is for the VND driver only, and in the
future, ACLs will prevent any other process from using this call.
When installed, the test scripts lose their ".sh" suffix, causing them
to be skipped by the "run" script. With this patch, the tests are no
longer specified with ".sh" suffix in the run script, and the suffix
is added automatically as necessary.
LSC: Minor adaptation to keep track of history. As this patch has been
forward ported into mainline, a simple rebase would loose this
commit (cf commit 1f317d315c8140c2bcdbff1953a6f645f4f6ef04).
- fail SEF initialization if any of the subtests failed, so that the
party invoking the "service up" can tell whether the test succeeded;
- add "nocontig" option, because VM isn't particularly good at
allocating contiguous memory;
- add "silent" option, because it floods the console otherwise;
- allow the device size to be smaller than the maximum transfer size;
- install files to installed test directory.
Not all services involved in block I/O go through VM to access the
blocks they need. As a result, the blocks in VM may become stale,
possibly causing corruption when the stale copy is restored by a
service that does go through VM later on. This patch restores support
for forgetting cached blocks that belong to a particular device, and
makes the relevant file systems use this functionality 1) when
requested by VFS through REQ_FLUSH, and 2) upon unmount.
Previously, VFS would reopen a character device after a driver crash
if the associated file descriptor was opened with the O_REOPEN flag.
This patch removes support for this feature. The code was complex,
full of uncovered corner cases, and hard to test. Moreover, it did not
actually hide the crash from user applications: they would get an
error code to indicate that something went wrong, and have to decide
based on the nature of the underlying device how to continue.
- remove support for O_REOPEN, and make playwave(1) reopen its device;
- remove support for the DEV_REOPEN protocol message;
- remove all code in VFS related to reopening character devices;
- no longer change VFS filp reference count and FD bitmap upon filp
invalidation; instead, make get_filp* fail all calls on invalidated
FDs except when obtained with the locktype VNODE_OPCL which is used
by close_fd only;
- remove the VFS fproc file descriptor bitmap entirely, returning to
the situation that a FD is in use if its slot points to a filp; use
FILP_CLOSED as single means of marking a filp as invalidated.
- block the calling thread on character device close;
- fully separate block and character open/close routines;
- reuse generic open/close code for the cloning case;
- zero all messages to drivers before filling them;
- use appropriate types for major/minor device numbers.
- prefix them with VFS_ as they are going to VFS;
- give these calls normal call numbers;
- give them their own set of message field aliases;
- also make do_mapdriver a regular call.
Lionel Sambuc [Wed, 2 Oct 2013 08:56:24 +0000 (10:56 +0200)]
ARM serial driver: Comment termios_baud_rate.
The B0-B115200 defines are flags, and not the actual speed they
represent.
This fixes an incoherency for B0 handling, and documents why it is
required to call the function again after changing the speed flag.
DFL_BAUD is set to one of the flag, so to translate it to an actual
speed, the function calls itself again, which will always be able to
finish without inducing another recursive call.
It is unclear why /dev/log has its own open/close rules, but those
rules conflict with serial console redirection. This does not solve
the root of the problem, but it puts back in place more or less the
same workaround that was already in place before the TTY overhaul.
- writing to a PTY master side blocks if there is not already a
blocked reader on the slave side, and select now reflects this;
- internally, TTY now uses a test based on "caller != NONE" rather
than "grant != GRANT_INVALID" to identify whether a call is
currently ongoing;
- "offset" fields have been removed as they equal the corresponding
"cum" fields;
- improved variable typing and function naming here and there;
- various other small fixes.
- check each file descriptor's open access mode (filp_mode);
- treat an error returned by a character driver as a select error;
- check all filps in each set before finishing select;
- do not copy back file descriptor sets if an error occurred;
- remove the hardcoded list of supported character major devices,
since all drivers should now be capable of responding properly;
- add tests to test40 and fix its error count aggregation.
- simplify and repair UDS request handling state machine;
- simplify interface used between internal modules;
- implement missing support for nonblocking I/O;
- fix select implementation;
- clean up global variables.
The new API now covers the entire character driver protocol, while
hiding all the message details. It should therefore be used by all
new character drivers. All existing drivers that already made use of
libchardriver have been changed to use the new API.
As one of the most important API changes, support for scatter and
gather transfers has been removed, as several key drivers already
did not support this, and it could be supported at the safecopy
level instead (for a future readv/writev).
Additional changes include:
- respond to block device open requests to avoid hanging VFS threads;
- add support for sef_cancel.
Some block drivers do not impose any alignment requirements, and this
patch allows such block drivers to pass the test set. As a side effect,
minimal support for min_write is added, but this part of blocktest is
in need of further improvement.
If a device node is given without path, and opening the node fails
initially, prepend "/dev/" to the node name and try opening again.
This is more in line with NetBSD behavior.
The block driver protocol and libblockdriver's bdr_ioctl hook are
changed, as well as the users of this hook. Other parts of the system
are expected to change accordingly eventually, since the ioctl(2)
prototype has been aligned with NetBSD's.
Block protocol: add user endpoint to IOCTL request
I/O control requests now come with the endpoint of the user process
that initiated the ioctl(2) call. It is stored in a new BDEV_USER
field, which is an alias for BDEV_FLAGS. The contents of this field
are to be used only in highly specific situations. It should be
preserved (not replaced!) by services that forward IOCTL requests,
and may be set to NONE for service-initiated IOCTL requests.
The original R_BIT and W_BIT definitions have nothing to do with the
way these bits are used. Their distinct usage is more apparent when
they have different names.
This constant determines the range of valid device_id_t values that
a block driver can return from the bdr_device hook: a value between
0 and (BLOCKDRIVER_MAX_DEVICES - 1) inclusive.
- internal structure rearrangement;
- respond to char device open requests to avoid hanging VFS threads;
- make drivers use designated initializers;
- use devminor_t for all minor device numbers;
- change bdr_other hook to take ipc_status and return nothing;
- fix default geometry computation;
- add support for sef_cancel.
Previously, reading from or writing to a character device would not
update the file position on the corresponding filp object. Performing
this update correctly is not trivial: during and after the I/O
operation, the filp object must not be locked. Ideally, read/write
requests on a filp that is already involved in a read/write operation,
should be queued. For now, we optimistically update the file position
at the start of the I/O; this works under the assumptions listed in
the corresponding comment.
Previously it would use bits of the character driver protocol, which
will change heavily. In the new situation, the BUSC_I2C_xxx requests
use a protocol more in line with the PCI protocol, with the reply code
in m_type.
Opening and closing the master side of a pseudo terminal without
opening the slave side would result in the pseudo terminal becoming
permanently unavailable. In addition, reopening the slave side
would be possible but not allow for I/O. Finally, attempting to
open an in-use master would wipe its I/O state. These issues have
been resolved.
POSIX states that when interrupted, partially successful pipe
operations should return the partial result rather than EINTR. VFS
previously wouldn't look at the partial result, and not clear it
either, which would result in a panic upon the next pipe operation.
The main purpose of this patch is to fix handling of unpause calls
from PM while another call is ongoing. The solution to this problem
sparked a full revision of the threading model, consisting of a large
number of related changes:
- all active worker threads are now always associated with a process,
and every process has at most one active thread working for it;
- the process lock is always held by a process's worker thread;
- a process can now have both normal work and postponed PM work
associated to it;
- timer expiry and non-postponed PM work is done from the main thread;
- filp garbage collection is done from a thread associated with VFS;
- reboot calls from PM are now done from a thread associated with PM;
- the DS events handler is protected from starting multiple threads;
- support for a system worker thread has been removed;
- the deadlock recovery thread has been replaced by a parameter to the
worker_start() function; the number of worker threads has
consequently been increased by one;
- saving and restoring of global but per-thread variables is now
centralized in worker_suspend() and worker_resume(); err_code is now
saved and restored in all cases;
- the concept of jobs has been removed, and job_m_in now points to a
message stored in the worker thread structure instead;
- the PM lock has been removed;
- the separate exec lock has been replaced by a lock on the VM
process, which was already being locked for exec calls anyway;
- PM_UNPAUSE is now processed as a postponed PM request, from a thread
associated with the target process;
- the FP_DROP_WORK flag has been removed, since it is no longer more
than just an optimization and only applied to processes operating on
a pipe when getting killed;
- assignment to "fp" now takes place only when obtaining new work in
the main thread or a worker thread, when resuming execution of a
thread, and in the special case of exiting processes during reboot;
- there are no longer special cases where the yield() call is used to
force a thread to run.
The T_DUMPCORE implementation was not only broken - it would currently
produce a coredump of the tracer process rather than the traced
process - but also deeply flawed, and fixing it would require serious
alteration of PM's internal state machine. It should be possible to
implement the same functionality in userland, and that is now the
suggested way forward. For now, also remove the (identical) utilities
using T_DUMPCORE: dumpcore(1) and gcore(1).
Previously, processing of some replies coming from character drivers
could block on locks, and therefore, such processing was done from
threads that were associated to the character driver process. The
hidden consequence of this was that if all threads were in use, VFS
could drop replies coming from the driver. This patch returns VFS to
a situation where the replies from character drivers are processed
instantly from the main thread, by removing the situations that may
cause VFS to block while handling those replies.
- change the locking model for select, so that it will never block
on any processing that happens after the select call has been set
up, in particular processing of character driver select replies;
- clearly mark all select routines that may never block;
- protect against race conditions in do_select as result of the
locking that still does happen there (as is required for pipes);
- also handle select timers from the main thread;
- move processing of character driver replies into device.c.
These days, DEV_OPEN calls to character drivers block the calling
thread until completion or failure, and thus never return SUSPEND to
the caller. The same already applied to BDEV_OPEN calls to block
drivers. It has thus become impossible for a process to enter a state
of being blocked on a device open call.
There is currently no support for restarting device open calls to
restarted character drivers. This support was present in the _DOPEN
logic, but was already no longer triggering. In the future, this case
should be handled by the thread performing the open request.
Not doing so caused PFS to commit protocol violations by relying on
stale information when sending replies. This stale information always
happened to be correct, which is why the problem went unnoticed.
Change-Id: Ia42ca670718d6e731193cd2c34a3ff455f8a94d3
- change all sync char drivers into async drivers;
- retire support for the sync protocol in libchardev;
- remove async dev style, as this is now the default;
- remove dev_status from VFS;
- clean up now-unused protocol messages.
The async char protocol already has this, so this patch closes the
gap between the two protocols a bit. Support for this flag has been
added to all sync char drivers that support CANCEL at all.
The LOG driver was already using the asynchronous protocol, but it
did not support the nonblocking transfer flag. This has been fixed
as well.
As with w_task, this ensures that the field remains cleared if it is
not used. Without this, worker_stop could mistakenly identify a thread
as talking to a device driver rather than a (crashed) file server.
Thomas Veerman [Sat, 24 Aug 2013 10:23:41 +0000 (12:23 +0200)]
VFS: set w_task only when needed
It was always set, but not always cleared, when talking to asynchronous
drivers. This could cause erratic behavior upon a driver crash.
Normally, a worker thread's w_task field is set when it's about to
communicate with a driver or FS. Then upon receiving a reply we can
do sanity checks (that the thread we want to wake up was actually
waiting for a reply). Also, when a driver/FS crashes, we can identify
which worker threads were talking to the crashed endpoint and handle
the error gracefully.
Asynchronous drivers are a bit special, though. In most cases, the
sender of the request is not interested in the reply (the sender was
suspended and only wants to know whether the request was successfully
caried out or not). However, the open request is special, as the reply
carries information needed by the sender. This is the only request
where a worker thread actually yields and waits for the result. This is
also the only case where we're interested in setting w_task for
asynchronous drivers.
* Removed startup code patches in lib/csu regarding kernel to userland
ABI.
* Aligned stack layout on NetBSD stack layout.
* Generate valid stack pointers instead of offsets by taking into account
_minix_kerninfo->kinfo->user_sp.
* Refactored stack generation, by moving part of execve in two
functions {minix_stack_params(), minix_stack_fill()} and using them
in execve(), rs and vm.
* Changed load offset of rtld (ld.so) to:
execi.args.stack_high - execi.args.stack_size - 0xa00000
which is 10MB below the main executable stack.
The main motivation for this change is that only Loris supports
multithreading, and Loris supports dynamic thread allocation, so the
number of supported threads can be implemented as a bit flag (i.e.,
either 1 or "at least as many as VFS has"). The ABI break obviates the
need to support file system versioning at this time, and several
other aspects are better implemented as flags as well. Other changes:
- replace peek/bpeek test upon mount with FS flag as well;
- mark libsffs as 64-bit file size capable;
- remove old (3.2.1) getdents support.
- pass in file system type through mount(2), and return this type in
statvfs structures as generated by [f]statvfs(2);
- align mount flags field with NetBSD's, splitting out service flags
which are not to be passed to VFS;
- remove limitation of mount ABI to 16-byte labels, so that labels
can be made larger in the future;
- introduce new m11 message union type for mount(2) as side effect.