Add Ceph status report submitted by wjw@digiware.nl.
Reviewed by: wblock Sponsored by: iXsystems
This commit is contained in:
Dru Lavigne
parent
3a5dfb4ac3
commit
f2fc5b580a
Notes:
svn2git
2020-12-08 03:00:23 +00:00
svn path=/head/; revision=49066
1 changed files with 168 additions and 0 deletions
|
|
@ -680,4 +680,172 @@
|
|||
Microsoft
|
||||
</sponsor>
|
||||
</project>
|
||||
|
||||
<project cat='proj'>
|
||||
<title>Ceph on FreeBSD</title>
|
||||
|
||||
<contact>
|
||||
<person>
|
||||
<name>
|
||||
<given>Willem Jan</given>
|
||||
<common>Withagen</common>
|
||||
</name>
|
||||
<email>wjw@digiware.nl</email>
|
||||
</person>
|
||||
</contact>
|
||||
|
||||
<links>
|
||||
<url href="http://ceph.com">Ceph main site</url>
|
||||
<url href="https://github.com/ceph/ceph">Main repository</url>
|
||||
<url href="https://github.com/wjwithagen/ceph">My Fork </url>
|
||||
<url href="https://github.com/ceph/ceph/pull/7573">The git PULL with all changes</url>
|
||||
</links>
|
||||
|
||||
<body>
|
||||
<p>Ceph is a distributed object store and file system designed
|
||||
to provide excellent performance, reliability, and
|
||||
scalability. It provides the following features:</p>
|
||||
|
||||
<ol>
|
||||
<li>Object Storage: Ceph provides seamless access to objects
|
||||
using native language bindings or <tt>radosgw</tt>, a REST
|
||||
interface that is compatible with applications written for
|
||||
S3 and Swift.</li>
|
||||
|
||||
<li>Block Storage: Ceph’s RADOS Block Device (RBD) provides
|
||||
access to block device images that are striped and
|
||||
replicated across the entire storage cluster.</li>
|
||||
|
||||
<li>File System: Ceph provides a POSIX-compliant network file
|
||||
system that aims for high performance, large data storage,
|
||||
and maximum compatibility with legacy applications.</li>
|
||||
</ol>
|
||||
|
||||
<p>I started looking into Ceph as using HAST with CARP and
|
||||
<tt>ggate</tt> did not meet my requirements. My primary goal
|
||||
with Ceph is to run a storage cluster of ZFS storage nodes
|
||||
where the clients run bhyve on RBD disks stored in Ceph.</p>
|
||||
|
||||
<p>The &os; build process can build most of the tools in
|
||||
Ceph. However, the RBD-dependent items do not work since
|
||||
&os; does not yet provide RBD support.</p>
|
||||
|
||||
<p>Since the last quarterly report, the following progress was
|
||||
made:</p>
|
||||
|
||||
<ol>
|
||||
<li>The changeover from using CMake to Automake results in a
|
||||
much cleaner development environment and better test output.
|
||||
The changes can be found in the
|
||||
<tt>wip-wjw-freebsd-cmake</tt> branch.</li>
|
||||
|
||||
<li>Throttling code has been overhauled to prevent live locks.
|
||||
These mainly occur on &os; but also manifest on Linux.</li>
|
||||
|
||||
<li>Fixed a few more tests. On one occasion, I was able to
|
||||
complete the full test set without errors.</li>
|
||||
</ol>
|
||||
|
||||
<p>11-CURRENT is used to compile and build test Ceph. The
|
||||
Clang toolset needs to be at least version 3.7 as Clang 3.4
|
||||
does not have all of the capabilities required to compile
|
||||
everything.</p>
|
||||
|
||||
<p>This setup will get things running for &os;:</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<p>Install <tt>bash</tt> and link it in <tt>/bin</tt>
|
||||
(requires root privileges):</p>
|
||||
|
||||
<p><tt>sudo pkg install bash</tt></p>
|
||||
|
||||
<p><tt>sudo ln -s /usr/local/bin/bash /bin/bash</tt></p>
|
||||
</li>
|
||||
|
||||
<li>
|
||||
<p>Build Ceph:</p>
|
||||
|
||||
<p><tt>git clone https://github.com/wjwithagen/ceph.git</tt></p>
|
||||
|
||||
<p><tt>cd ceph</tt></p>
|
||||
|
||||
<p><tt>git checkout wip-wjw-freebsd-tests</tt></p>
|
||||
|
||||
<p><tt>./do_freebsd.sh</tt></p>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<p>Parts Not Yet Included:</p>
|
||||
|
||||
<ul>
|
||||
<li>RBD: Rados Block Devices are currently implemented in the
|
||||
Linux kernel but there used to be a userspace
|
||||
implementation. It is possible that <tt>ggated</tt> could
|
||||
be used as a template since it provides some of the same
|
||||
functionality and it has a userspace counterpart.</li>
|
||||
|
||||
<li>BlueStore: &os; and Linux have a different AIO API which
|
||||
needs to be made compatible. There has been some
|
||||
discussion about <tt>aio_cancel</tt> not working for all
|
||||
device types in &os;.</li>
|
||||
|
||||
<li>CephFS: Cython tries to access an internal field in dirent
|
||||
which does not compile.</li>
|
||||
|
||||
<li>Tests that verify the correct working of the above are
|
||||
also excluded from the testset.</li>
|
||||
</ul>
|
||||
|
||||
<p>Tests Not Yet Included:</p>
|
||||
|
||||
<ul>
|
||||
<li><tt>ceph-detect-init/run-tox.sh</tt>: the current
|
||||
implementation does not know anything about &os;
|
||||
<tt>rc-init</tt>.</li>
|
||||
|
||||
<li>Tests that make use of <tt>nosestests</tt> do not really
|
||||
work since <tt>nostests</tt> is not in <tt>/usr/bin</tt>,
|
||||
and calling <tt>/usr/bin/env/nosetests</tt> does not work on
|
||||
&os;.</li>
|
||||
|
||||
<li><tt>test/pybind/test_ceph_argparse.py</tt></li>
|
||||
|
||||
<li><tt>test/pybind/test_ceph_daemon.py</tt></li>
|
||||
</ul>
|
||||
</body>
|
||||
|
||||
<help>
|
||||
<task>The current and foremost task it to get the test set to
|
||||
complete without errors.</task>
|
||||
|
||||
<task>Build an automated test platform that will build
|
||||
<tt>ceph/master</tt> on &os; and report the results back to
|
||||
the Ceph developers. This will increase the maintainability
|
||||
of the &os; side of things as developers are signaled that
|
||||
they are using Linux-isms that will not compile or run on
|
||||
&os;. Ceph has several projects that support this: Jenkins,
|
||||
teuthology, and palpito. But even a
|
||||
<tt>while { compile }</tt> loop that reports the build data on
|
||||
a static webpage is a good start.</task>
|
||||
|
||||
<task>Run integration tests to see if the &os; daemons will work
|
||||
with a Linux Ceph platform.</task>
|
||||
|
||||
<task>Get the currently excluded Python tests to work.</task>
|
||||
|
||||
<task>Compile and test the user space RBD (Rados Block
|
||||
Device).</task>
|
||||
|
||||
<task>Investigate if an in-kernel RBD device could be developed
|
||||
ala <tt>ggate</tt>.</task>
|
||||
|
||||
<task>Investigate the keystore which currently prevents the
|
||||
building of Cephfs and some other parts.</task>
|
||||
|
||||
<task>Integrate the &os; <tt>/etc/rc.d</tt> init scripts in the
|
||||
Ceph stack for testing and for running Ceph on production
|
||||
machines.</task>
|
||||
</help>
|
||||
</project>
|
||||
</report>
|
||||
|
|
|
|||
Loading…
Reference in a new issue