Ben Gras [Tue, 7 May 2013 12:36:09 +0000 (12:36 +0000)]
vm: mmap support
. test74 for mmap functionality
. vm: add a mem_file memory type that specifies an mmap()ped
memory range, backed by a file
. add fdref, an object that keeps track of FD references within
VM per process and so knows how to de-duplicate the use of FD's
by various mmap()ped ranges; there can be many more than there can
be FD's
. turned off for now, enable with 'filemap=1' as boot option
Ben Gras [Tue, 7 May 2013 12:41:07 +0000 (12:41 +0000)]
vfs: mmap support
. libc: add vfs_mmap, a way for vfs to initiate mmap()s.
This is a good special case to have as vfs is a slightly
different client from regular user processes. It doesn't do it
for itself, and has the dev & inode info already so the callback
to VFS for the lookup isn't necessary. So it has different info
to have to give to VM.
. libc: also add minix_mmap64() that accepts a 64-bit offset, even
though our off_t is still 32 bit now.
. On exec() time, try to mmap() in the executable if available.
(It is not yet available in this commit.)
. To support mmap(), add do_vm_call that allows VM to lookup
(to ino+dev), do i/o from and close FD's on behalf of other
processes.
build:arm allow to store build settings in a .settings file.
If present a file called .settings will be sourced when
calling the arm_sdimage.sh script. This allows to configure
the build setting for arm build without the need to specify
the flags each time arm_sdimage.sh is called.
For example to configure arm_sdimage.sh to target the
beaglebone put the following content in your .settings file.
Thomas Cort [Tue, 28 May 2013 19:07:10 +0000 (15:07 -0400)]
syslog.conf: correct PID file path in comments.
The syslogd.pid file has moved from /usr/run to /var/run. The comments
in syslog.conf direct the user to restart syslogd using the old PID
file location. This patch updates the comment to match the code.
Omap timers remove hardcoded base address and add some initial
support for the beaglebone's timers. Frclock_util will need
refactoring to remain independent of the ARM flavour.
We are planning on using the NetBSD config system to perform build
system configuration when targeting different hardware variants.
This functionality however is not integrated yet and we currently
rely on using ifdef's in the code. This commit add the DDM37XX flag.
Lionel Sambuc [Thu, 23 May 2013 16:09:19 +0000 (18:09 +0200)]
releasetools/arm_sdimage.sh: Improvements
- Added the possibility to define the number of parallel jobs from the
command line with (example, default is 1):
$ JOBS=8 ./releasetools/arm_sdimage.sh
- Generate filesystem filling the reserved partition area, instead of
being smaller.
- Added a possibility to specify additional parameter to the build.sh
script from the command line with (example):
$ BUILDVARS='-N0 -V MYVAR=yes' ./releasetools/arm_sdimage.sh
Lionel Sambuc [Thu, 23 May 2013 16:00:34 +0000 (18:00 +0200)]
Fix symlink issue when generating an ARM image
A few symlinks were pointing to a nonexistent file named '(null)'
instead of their intended target. This was only seen when using the
arm_sdimage.sh script.
There is two ways of specifying links and directories. the first one
using an entry in one of the mtree files, and the second one in
/etc/Makefile.
Those entries where doubled, and one of them would specify some rights,
while the other had the target.
By removing those entries, I make sure there is only one definition of
these symlinks, which solves the problem I was seeing on an ARM image.
These symlinks are still present on a generated system, as they are
required.
Ben Gras [Thu, 23 May 2013 00:43:33 +0000 (02:43 +0200)]
mkfs: only complain about failing stat with -d
mkfs -d is a feature that needs a stat() of the mkfs binary, which
often fails (as mkfs is often not invoked with a full path or from
the same directory). it makes setup look a bit messy as the error
is always printed while installing the system, for each created FS,
even though the situation is harmless. This change only complains
when the stat is actually needed (-d).
Ben Gras [Tue, 21 May 2013 12:57:57 +0000 (14:57 +0200)]
arm: recognize instruction fetch pagefault
. an instruction fetch pagefault generates a prefetch
abort exception, this should also be a source of
a pagefault event
. the saved_lr argument to the C exception handler was
sometimes an address, sometimes a pointer to that address;
the kernel sometimes dereferences it, so it should always be
an in-kernel address, never a userspace address. fix in mpx.S
makes it always a pointer.
. move dumping of all processes over serial out of
the arch-specific arch_system.c
Antoine Leca [Fri, 26 Apr 2013 18:54:22 +0000 (20:54 +0200)]
mkfs reloaded
* restored the zone>block stuff
* generalized the support for more-than-1-block bitmaps to also deal
with big inodes maps in small blocks
* improved pexit() to printf-like syntax; also replaced pexit() with
err() or errx() where there are no reference to proto lines
* unified the allocation of blocks to all use alloc_block() instead
of raw malloc half of the time
* removed the V2_ d2_ etc. prefixes which are obscure and obsolete
While here, also
* used new-form definitions, and closer to KNF style in general
* used often-built-ins such as mem* or strncpy where relevant
* pruned a fair amount of constants which are irrelevant
Thanks to Thomas V. and Lionel who did a good amount of work with the
cross-compilation stuff and made that change to be much smoother.
Ben Gras [Wed, 1 May 2013 19:02:06 +0000 (19:02 +0000)]
kernel ipc debug: various fixes
. add receive hooks in the kernel to print asynchronously
delivered messages
. do not rely on MF_REPLY_PEND to decide between calls and errors,
as that isn't reliable for asynchronous messages; try both instead
. add _sendcall() that extract-mfield.sh can then reliably recognize
the fields for messages that are sent with just send()
. add DEBUG_DUMPIPC_NAMES to restrict printed messages to
from/to given process names
Ben Gras [Fri, 26 Apr 2013 09:54:33 +0000 (09:54 +0000)]
libminixfs, mfs, ext2: may re-evaluate cache size
libminixfs may now be informed of changes to the block usage on the
filesystem. if the net change becomes big enough, libminixfs may
resize the cache based on the new usage.
. update the 2 FSes to provide this information to libminixfs
Ben Gras [Sat, 16 Mar 2013 04:09:36 +0000 (04:09 +0000)]
VFS: Implement REQ_BPEEK.
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS. Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.
Ben Gras [Sat, 16 Mar 2013 03:46:37 +0000 (03:46 +0000)]
libexec: mmap support, prealloc variants
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.
Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.
Ben Gras [Sat, 16 Mar 2013 02:29:32 +0000 (02:29 +0000)]
MFS: annotate cache blocks with inode metadata
. use lmfs_* cache functions that provide the cache with inode
metadata whenever applicable, i.e. tell the cache code which
inode number and in-inode offset a particular cache block
corresponds to.
. needed for mmap implementation
Ben Gras [Wed, 20 Mar 2013 19:18:52 +0000 (19:18 +0000)]
vm: new secondary cache code
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e. also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.
This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.
This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.
Ben Gras [Wed, 20 Mar 2013 19:09:01 +0000 (19:09 +0000)]
VM: memtype fix
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.
Ben Gras [Wed, 20 Mar 2013 18:45:10 +0000 (18:45 +0000)]
VM: 64-bit mmap()
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).
Ben Gras [Wed, 20 Mar 2013 17:04:39 +0000 (17:04 +0000)]
filesystems: return ENOSYS for REQ_PEEK
The filesystems already implement REQ_PEEK, but do not fully
use the new filesystem cache code yet. (Because it isn't committed
yet..) REQ_PEEK should be disabled for them until they do.
This indicates to VFS that they are not annotating their cache
blocks (in VM) with inode number/offset info, and therefore mmap()
shouldn't succeed on any of their files. (Most importantly exec()
won't fallback elegantly otherwise.)
build:refactor arm_sdimage.sh as to no longer require root.
There where two reasons for needing root access while making ARM builds. The
first reason was that we needed root permissions to create device nodes and
assign permissions to files. The second reason we needed root was that we used
loop mounted files to copy the build output to the file backed storage.
The first problem is solved by building in unprivileged mode (adding -U) during
compilation. The second problem is solved by using mcopy to put files on the
fat partition and use the mkfs.mfs proto files to create MFS file system.
Future work might be to replace this script by the netbsd makefs tool.
Ben Gras [Fri, 12 Apr 2013 20:54:42 +0000 (20:54 +0000)]
many new tests
. test70: regression test for m_out vfs race condition
The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.
. test71: blackbox full-stack test of FS operation, testing
using the regular VFS interface crazy i/o patterns
with various working set sizes, triggering only
primary cache, also secondary cache, and finally
disk i/o and verifying contents all the time
. test72: unit test of libminixfs, implementing
functions it needs from -lsys and -lblockdriver
and the client in order to simulate a working
cache client and backend environment.
. test73: blackbox test of secondary vm cache in isolation
Ben Gras [Mon, 15 Apr 2013 17:44:19 +0000 (17:44 +0000)]
vfs: fix null deref, pfs: add fchmod()
. vfs read_only() assumes vnode->v_vmnt is non-NULL, but it can
be NULL sometimes
. e.g. fchmod() on UDS triggered NULL deref; add a check and
add REQ_CHMOD to pfs so unix domain sockets can be fchmod()ded
. add to test56
Ben Gras [Thu, 18 Apr 2013 13:41:01 +0000 (13:41 +0000)]
test run: feature for nicer jenkins interaction
. add -l option to list available tests
. add -t option to specify tests to run
. also improve the root check a bit by
not relying on an environment var
. do not print the human-friendly banner & summary
if a test list is given to make the test
results easy to parse
Ben Gras [Tue, 16 Apr 2013 16:04:46 +0000 (16:04 +0000)]
tests cleanup
. make common.o link with the tests instead of being
#included as common.c
. fix warnings about missing prototypes by declaring functions
static
. reduces some duplicated code
Ben Gras [Fri, 12 Apr 2013 16:41:23 +0000 (16:41 +0000)]
vfs: make m_out non-global
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change
. makes m_out local to the message handling function,
declared on the stack of the caller
. forces callers of reply() to give it a message, or
declare the reply message has no significant fields except
for the return code by calling replycode()
Antoine Leca [Sat, 24 Dec 2011 14:02:54 +0000 (15:02 +0100)]
utimens(2) system call
Variant of utime(2) with struct timespec (with ns precision)
instead of time_t values; also allows for tv_nsec members
the values UTIME_NOW (force update to current time) or
UTIME_OMIT (allow to set either atim or mtim independently.)
Provides a superset of utimes(2), futimes(2), lutimes(2),
and futimens(2).
Provides the same subset of utimensat(2) as does NetBSD 6.
Also import utimens() and lutimeNS() from NetBSD-current.
The kernel API for requesting interrupts and the associated callback
have a somewhat strange behaviour. Requesting an interrupts is done
by calling sys_irqsetpolicy using an interrupt and a given id. This
id can be modified by the sys_irqsetpolicy and must be used for
subsequent calls to sys_irqenable/sys_irqdisable. However upon an
incoming call from the kernel NOTIFY_ARG contains the original value
encoded in a set e.g. if 1 << id == true the interrupt was raised.
We used to load the memory driver above the other modules to allow
the memory driver (ramdisk) to grow. We no longer want or need this
this as we have a working mmc driver.