Ben Gras [Fri, 26 Apr 2013 09:54:33 +0000 (09:54 +0000)]
libminixfs, mfs, ext2: may re-evaluate cache size
libminixfs may now be informed of changes to the block usage on the
filesystem. if the net change becomes big enough, libminixfs may
resize the cache based on the new usage.
. update the 2 FSes to provide this information to libminixfs
Ben Gras [Sat, 16 Mar 2013 04:09:36 +0000 (04:09 +0000)]
VFS: Implement REQ_BPEEK.
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS. Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.
Ben Gras [Sat, 16 Mar 2013 03:46:37 +0000 (03:46 +0000)]
libexec: mmap support, prealloc variants
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.
Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.
Ben Gras [Sat, 16 Mar 2013 02:29:32 +0000 (02:29 +0000)]
MFS: annotate cache blocks with inode metadata
. use lmfs_* cache functions that provide the cache with inode
metadata whenever applicable, i.e. tell the cache code which
inode number and in-inode offset a particular cache block
corresponds to.
. needed for mmap implementation
Ben Gras [Wed, 20 Mar 2013 19:18:52 +0000 (19:18 +0000)]
vm: new secondary cache code
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e. also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.
This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.
This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.
Ben Gras [Wed, 20 Mar 2013 19:09:01 +0000 (19:09 +0000)]
VM: memtype fix
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.
Ben Gras [Wed, 20 Mar 2013 18:45:10 +0000 (18:45 +0000)]
VM: 64-bit mmap()
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).
Ben Gras [Wed, 20 Mar 2013 17:04:39 +0000 (17:04 +0000)]
filesystems: return ENOSYS for REQ_PEEK
The filesystems already implement REQ_PEEK, but do not fully
use the new filesystem cache code yet. (Because it isn't committed
yet..) REQ_PEEK should be disabled for them until they do.
This indicates to VFS that they are not annotating their cache
blocks (in VM) with inode number/offset info, and therefore mmap()
shouldn't succeed on any of their files. (Most importantly exec()
won't fallback elegantly otherwise.)
build:refactor arm_sdimage.sh as to no longer require root.
There where two reasons for needing root access while making ARM builds. The
first reason was that we needed root permissions to create device nodes and
assign permissions to files. The second reason we needed root was that we used
loop mounted files to copy the build output to the file backed storage.
The first problem is solved by building in unprivileged mode (adding -U) during
compilation. The second problem is solved by using mcopy to put files on the
fat partition and use the mkfs.mfs proto files to create MFS file system.
Future work might be to replace this script by the netbsd makefs tool.
Ben Gras [Fri, 12 Apr 2013 20:54:42 +0000 (20:54 +0000)]
many new tests
. test70: regression test for m_out vfs race condition
The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.
. test71: blackbox full-stack test of FS operation, testing
using the regular VFS interface crazy i/o patterns
with various working set sizes, triggering only
primary cache, also secondary cache, and finally
disk i/o and verifying contents all the time
. test72: unit test of libminixfs, implementing
functions it needs from -lsys and -lblockdriver
and the client in order to simulate a working
cache client and backend environment.
. test73: blackbox test of secondary vm cache in isolation
Ben Gras [Mon, 15 Apr 2013 17:44:19 +0000 (17:44 +0000)]
vfs: fix null deref, pfs: add fchmod()
. vfs read_only() assumes vnode->v_vmnt is non-NULL, but it can
be NULL sometimes
. e.g. fchmod() on UDS triggered NULL deref; add a check and
add REQ_CHMOD to pfs so unix domain sockets can be fchmod()ded
. add to test56
Ben Gras [Thu, 18 Apr 2013 13:41:01 +0000 (13:41 +0000)]
test run: feature for nicer jenkins interaction
. add -l option to list available tests
. add -t option to specify tests to run
. also improve the root check a bit by
not relying on an environment var
. do not print the human-friendly banner & summary
if a test list is given to make the test
results easy to parse
Ben Gras [Tue, 16 Apr 2013 16:04:46 +0000 (16:04 +0000)]
tests cleanup
. make common.o link with the tests instead of being
#included as common.c
. fix warnings about missing prototypes by declaring functions
static
. reduces some duplicated code
Ben Gras [Fri, 12 Apr 2013 16:41:23 +0000 (16:41 +0000)]
vfs: make m_out non-global
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change
. makes m_out local to the message handling function,
declared on the stack of the caller
. forces callers of reply() to give it a message, or
declare the reply message has no significant fields except
for the return code by calling replycode()
Antoine Leca [Sat, 24 Dec 2011 14:02:54 +0000 (15:02 +0100)]
utimens(2) system call
Variant of utime(2) with struct timespec (with ns precision)
instead of time_t values; also allows for tv_nsec members
the values UTIME_NOW (force update to current time) or
UTIME_OMIT (allow to set either atim or mtim independently.)
Provides a superset of utimes(2), futimes(2), lutimes(2),
and futimens(2).
Provides the same subset of utimensat(2) as does NetBSD 6.
Also import utimens() and lutimeNS() from NetBSD-current.
The kernel API for requesting interrupts and the associated callback
have a somewhat strange behaviour. Requesting an interrupts is done
by calling sys_irqsetpolicy using an interrupt and a given id. This
id can be modified by the sys_irqsetpolicy and must be used for
subsequent calls to sys_irqenable/sys_irqdisable. However upon an
incoming call from the kernel NOTIFY_ARG contains the original value
encoded in a set e.g. if 1 << id == true the interrupt was raised.
We used to load the memory driver above the other modules to allow
the memory driver (ramdisk) to grow. We no longer want or need this
this as we have a working mmc driver.
This files defines two constants which are only used through
sys/syslimits. So they where moved there instead of including the file
and it was removed.
* Also cleanup a bit the makefile.
* Also removing small inconsistency for arm builds where log would be
skipped if doing a make hdboot, but not when cross-compiling.
Thomas Cort [Sat, 6 Apr 2013 22:46:56 +0000 (22:46 +0000)]
tests/run: add test69 to setuids.
test69 requires superuser powers to adjust the time. This caused
the test to fail when run as a normal user. The patch adds test69
to the setuid list which will allow regular users to execute it.
Thomas Cort [Sat, 6 Apr 2013 12:47:10 +0000 (12:47 +0000)]
Importing usr.sbin/rdate
Import the NetBSD rdate command and remove the Minix rdate command.
The default behaviour for both is the same. The NetBSD version adds
options to just display the time, adjust the time using adjtime(),
and set the time without printing the time.
Porting Notes:
- Compiles cleanly out of the box without any warnings
- Path changes from /usr/bin/rdate to /usr/sbin/rdate
- checked pkgsrc for any usages of rdate (none found)
- checked src for any usages of rdate (none found)
Testing:
- all command line options work (tested with time.nist.gov server)
- Native and cross build OK
Ben Gras [Fri, 5 Apr 2013 15:24:47 +0000 (15:24 +0000)]
top: a few fixes
. fixed overflow in ticks compare function, causing
occasionally esp. the kernel to be mis-ordered by cpu
time
. fixed some const string related warnings
. fixed some variable name shadowing warnings
Thomas Cort [Sat, 30 Mar 2013 16:59:21 +0000 (16:59 +0000)]
libc: add clock_settime() system call.
This also adds the sys_settime() kernel call which allows for the adjusting
of the clock named realtime in the kernel. The existing sys_stime()
function is still needed for a separate job (setting the boottime). The
boottime is set in the readclock driver. The sys_settime() interface is
meant to be flexible and will support both clock_settime() and adjtime()
when adjtime() is implemented later.
settimeofday() was adjusted to use the clock_settime() interface.
One side note discovered during testing: uptime(1) (part of the last(1)),
uses wtmp to determine boottime (not Minix's times(2)). This leads `uptime`
to report odd results when you set the time to a time prior to boottime.
This isn't a new bug introduced by my changes. It's been there for a while.
Thomas Cort [Fri, 29 Mar 2013 21:48:22 +0000 (21:48 +0000)]
libc: add clock_getres()/clock_gettime() system calls.
In order to make it more clear that ticks should be used for timers
and realtime should be used for timestamps / displaying the date/time,
getuptime() was renamed to getticks() and getuptime2() was renamed to
getuptime().
Servers, drivers, libraries, tests, etc that use getuptime()/getuptime2()
have been updated. In instances where a realtime was calculated, the
calculation was changed to use realtime.
System calls clock_getres() and clock_gettime() were added to PM/libc.
Thomas Cort [Fri, 29 Mar 2013 19:34:29 +0000 (19:34 +0000)]
kernel: rename realtime to monotonic, add realtime
Old realtime was used for both timers (where an accurate count of
all ticks is needed) and the system time. In order to implement
adjtime(2), these duties must be separated as changing the time
of day by a small amount shouldn't affect timers in any way nor
should it change the boot time.
Following the naming of the clocks used by clock_gettime(2). The
clock named 'realtime' will represent the best guess at the
current wall clock time, and the clock named 'monotonic' will
represent the absolute time the system has been running.
Use monotonic for timers in kernel and in drivers. Use realtime
for determining time of day, dates, etc.
This commit simply renames realtime to monotonic and adds a new
tick counter named realtime. There are no functional changes in
this commit. It just lays the foundation for future work.
Ben Gras [Tue, 2 Apr 2013 22:42:17 +0000 (22:42 +0000)]
some etc fixes
. split user-editable and system-owned files in etc/Makefile
. mtab is a symlink, not a file now; remove it
. force-install of certain system-controlled /etc files from
top Makefile
. rename /etc/make.conf to /etc/mk.conf; and don't set $ARCH;
reduce difference in bsd.own.mk
Thomas Cort [Sun, 17 Mar 2013 23:54:20 +0000 (19:54 -0400)]
release.sh: pkg_install should be installed 1st
pkgin now depends on pkg_install. If pkg_install appears after pkgin
in PREINSTALLED_PACKAGES, there will be an error because pkg_install
was installed as a dependency for pkgin.
Ben Gras [Sat, 29 Dec 2012 00:16:33 +0000 (00:16 +0000)]
VM: pagequeue data structure
. data structure that automatically keeps a set
of pages in reserve, to replace sparepages and
possibly re-used in the future for similar situations,
e.g. if in-filesystem-cache block eviction is
implemented and FS asks for a new block
Lionel Sambuc [Fri, 8 Mar 2013 18:18:09 +0000 (19:18 +0100)]
Install default users directly
Install /home/ast and /home/bin as part of the system build procedure,
as the setup script is not ran on the arm image.
/home/bin is needed for successful completion of our test suite.
This patch does not change the setup script, as it is not bothered by
those files/directiories being already there, and may be useful during
reinstallation on intel systems.
Thomas Veerman [Fri, 8 Mar 2013 14:17:13 +0000 (14:17 +0000)]
UDS: terminate canonical path string
When you provided a string with junk after the terminating nul to a
UNIX domain socket and used bind(2), the canonical path function would
not properly terminate the new string. This caused VFS to return
ENAMETOOLONG on an otherwise valid path name.
* Use the same output directory as was previously the standard.
* Add copying of uEnv and cmdline to the obj directory to allow booting from tftp.
* No longer start qemu at the end of the process.
* Use sudo if the script was called as non root user.