From nobody Thu Mar 28 13:35:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of ovirt.org designates 66.187.230.42 as permitted sender) client-ip=66.187.230.42; envelope-from=kimchi-devel-bounces@ovirt.org; helo=lists.ovirt.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of ovirt.org designates 66.187.230.42 as permitted sender) smtp.mailfrom=kimchi-devel-bounces@ovirt.org; Return-Path: Received: from lists.ovirt.org (lists.phx.ovirt.org [66.187.230.42]) by mx.zohomail.com with SMTPS id 1493744395220769.991062965645; Tue, 2 May 2017 09:59:55 -0700 (PDT) Received: from lists.phx.ovirt.org (localhost [127.0.0.1]) by lists.ovirt.org (Postfix) with ESMTP id 928FA82070C; Tue, 2 May 2017 16:59:53 +0000 (UTC) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lists.ovirt.org (Postfix) with ESMTPS id E1D9C8206E4 for ; Tue, 2 May 2017 16:59:21 +0000 (UTC) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v42Gt21I086767 for ; Tue, 2 May 2017 12:59:21 -0400 Received: from e24smtp01.br.ibm.com (e24smtp01.br.ibm.com [32.104.18.85]) by mx0b-001b2d01.pphosted.com with ESMTP id 2a6tn9c956-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 02 May 2017 12:59:20 -0400 Received: from localhost by e24smtp01.br.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 2 May 2017 13:59:18 -0300 Received: from d24relay02.br.ibm.com (9.18.232.42) by e24smtp01.br.ibm.com (10.172.0.143) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 2 May 2017 13:59:15 -0300 Received: from d24av05.br.ibm.com (d24av05.br.ibm.com [9.18.232.44]) by d24relay02.br.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v42GxEYa33489046 for ; Tue, 2 May 2017 13:59:15 -0300 Received: from d24av05.br.ibm.com (localhost [127.0.0.1]) by d24av05.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v42GxGXX016992 for ; Tue, 2 May 2017 13:59:16 -0300 Received: from alinefm-TP440.br.ibm.com (alinefm-TP440.br.ibm.com [9.18.239.173]) by d24av05.br.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v42GxGRA016979 for ; Tue, 2 May 2017 13:59:16 -0300 X-Original-To: kimchi-devel@ovirt.org From: Aline Manera To: Kimchi Devel Date: Tue, 2 May 2017 13:59:12 -0300 X-Mailer: git-send-email 2.9.3 X-TM-AS-MML: disable x-cbid: 17050216-1523-0000-0000-0000029AFEA9 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17050216-1524-0000-0000-00002A314690 Message-Id: <20170502165912.27169-1-alinefm@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-05-02_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=8 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705020094 Subject: [Kimchi-devel] [PATCH] [Wok] Add federation feature to Wok X-BeenThere: kimchi-devel@ovirt.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: kimchi-devel-bounces@ovirt.org Errors-To: kimchi-devel-bounces@ovirt.org X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Type: text/plain; charset="utf-8" It was primarily implemented on Kimchi project. But it makes more sense to be on Wok since the idea is to allow user add/select on which server Wok should manage. For example, the user could have a Wok installation on server A to manage server B. Signed-off-by: Aline Manera Reviewed-by: Daniel Barboza --- docs/API/config.md | 2 ++ docs/API/peers.md | 20 +++++++++++ docs/README-federation.md | 60 ++++++++++++++++++++++++++++++++ docs/wokd.8.in | 6 +++- src/wok.conf.in | 4 +++ src/wok/config.py.in | 1 + src/wok/control/peers.py | 30 ++++++++++++++++ src/wok/model/config.py | 1 + src/wok/model/peers.py | 74 ++++++++++++++++++++++++++++++++++++= +++ src/wokd.in | 5 +++ tests/test_api.py | 6 +++- tests/test_config_model.py | 2 +- tests/test_server_root.py | 6 ++-- ui/js/src/wok.api.js | 14 ++++++++ ui/js/wok.peers.js | 75 ++++++++++++++++++++++++++++++++++++= ++++ ui/pages/tabs/settings.html.tmpl | 34 +++++++++++++++++- 16 files changed, 333 insertions(+), 7 deletions(-) create mode 100644 docs/API/peers.md create mode 100644 docs/README-federation.md create mode 100644 src/wok/control/peers.py create mode 100644 src/wok/model/peers.py create mode 100644 ui/js/wok.peers.js diff --git a/docs/API/config.md b/docs/API/config.md index d1d1007..0e47f5f 100644 --- a/docs/API/config.md +++ b/docs/API/config.md @@ -12,6 +12,8 @@ Contains information about the application environment an= d configuration. * proxy_port: SSL port to list on * websockets_port: Port for websocket proxy to listen on * auth: Authentication method used to log in to Wok + * server_root: Relative path to Wok server. No value is specified by d= efault, ie, Wok will run on '/' + * federation: 'on' if federation feature is enabled, 'off' otherwise. * version: Wok version * **POST**: *See Task Actions* =20 diff --git a/docs/API/peers.md b/docs/API/peers.md new file mode 100644 index 0000000..fa5cf41 --- /dev/null +++ b/docs/API/peers.md @@ -0,0 +1,20 @@ +## REST API Specification for Peers + +### Collection: Peers + +**URI:** /peers + +Return a list of Wok peers in the same network. +(It uses openSLP for discovering) + +**Methods:** + +* **GET**: Retrieve a list peers URLs. + +#### Examples +GET /peers +[ + https://wok-peer0:8001, + https://wok-peer1:8001, + https://wok-peer2:8001, +] diff --git a/docs/README-federation.md b/docs/README-federation.md new file mode 100644 index 0000000..18e19c7 --- /dev/null +++ b/docs/README-federation.md @@ -0,0 +1,60 @@ +Wok Project - Federation Feature +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Federation feature is a mechanism to discover Wok peers in the same +network. It uses openSLP tool (http://www.openslp.org/) to register and fi= nd Wok +servers. + +By default this feature is disabled on Wok as it is not critical for KVM +virtualization and requires additional software installation. + +To enable it, do the following: + +1. Install openslp and openslp-server rpm packages, + or install slpd and slptool deb packages. + +2. openSLP uses port 427 (UDP) and port 427 (TCP) so make sure to open tho= se + ports in your firewall configuration + + For system using firewalld, do: + sudo firewall-cmd --permanent --add-port=3D427/udp + sudo firewall-cmd --permanent --add-port=3D427/tcp + sudo firewall-cmd --reload + + For openSUSE systems, do: + sudo /sbin/SuSEfirewall2 open EXT TCP 427 + sudo /sbin/SuSEfirewall2 open EXT UDP 427 + + For system using iptables, do: + sudo iptables -A INPUT -p tcp --dport 427 -j ACCEPT + sudo iptables -A INPUT -p udp --dport 427 -j ACCEPT + +3. In addition to the openSLP ports, you also need to allow multicast in t= he + firewall configuration + + For system using firewalld, do: + sudo firewall-cmd --direct --add-rule ipv4 filter INPUT 0 -s -= j ACCEPT + + For openSUSE systems, do: + Add the subnet to the trusted networks listed on FW_TRUSTED_NETS in + /etc/sysconfig/SuSEfirewall2 file. + Make sure to restart /sbin/SuSEfirewall2 after modifying /etc/sysconfig= /SuSEfirewall2 + + For system using iptables, do: + sudo iptables -A INPUT -s -j ACCEPT + +4. Start slpd service and make sure it is up while running Wok + sudo service slpd start + +5. Enable federation on Wok by editing the /etc/wok/wok.conf file: + + federation =3D on + +6. Then start Wok service + sudo service wokd start + +The Wok server will be registered on openSLP on server starting up and will +be found by other Wok peers (with federation feature enabled) in the same +network. + +Enjoy! diff --git a/docs/wokd.8.in b/docs/wokd.8.in index 217d9f6..09b5847 100644 --- a/docs/wokd.8.in +++ b/docs/wokd.8.in @@ -7,7 +7,7 @@ wokd - Launch Wok web server [\fB--cherrypy_port\fP \fICHERRYPY_PORT\fP] [\fB--websockets_port\fP \fIWE= BSOCKETS_PORT\fP] [\fB--session_timeout\fP \fISESSION_TIMEOUT\fP] [\fB--log_level\fP \fILOG_= LEVEL\fP] [\fB--log_dir\fP \fILOG_DIR\fP] [\fB--environment\fP \fIENV\fP] -[\fB--server_root\fP \fISERVER_ROOT\fP] [\fB--test\fP] +[\fB--server_root\fP \fISERVER_ROOT\fP] [\fB--federation\fP \fIfederation\= fP] [\fB--test\fP] .SH DESCRIPTION \fBWok\fP is a cherrypy-based web framework with HTML5 support originated = from Kimchi. It can be extended by plugins which expose functionality through REST APIs. @@ -55,6 +55,10 @@ Check cherrypy documentation for more details (default \= fIproduction\fP). \fB\-\-server_root\fP [\fISERVER_ROOT\fP] Relative path to Wok server. No value is specified by default, ie, Wok wil= l run on '/'. .TP +\fB\-\-federation\fP [\fIon\fP | \fIoff\fP] +Register and discover Wok peers in the same network using OpenSLP. Check +below the \fBREAD-federation\fP for more details (default \fIoff\fP). +.TP \fB\-\-test\fP Run Wok on a mock version that does not affect the system. For testing pro= posals. It depends on how plugins implements the mock environment as well. diff --git a/src/wok.conf.in b/src/wok.conf.in index 1ebdacf..1ed6310 100644 --- a/src/wok.conf.in +++ b/src/wok.conf.in @@ -27,6 +27,10 @@ # uncomment the following: #server_root=3D/wok =20 +# Federation feature: register Wok server on openSLP and discover peers +# in the same network. Check README-federation for more details. +#federation =3D off + [logging] # Log directory =20 diff --git a/src/wok/config.py.in b/src/wok/config.py.in index bbf2c09..39d7ad2 100644 --- a/src/wok/config.py.in +++ b/src/wok/config.py.in @@ -277,6 +277,7 @@ def _get_config(): config.set("server", "environment", "production") config.set('server', 'max_body_size', '4*1024*1024') config.set("server", "server_root", "") + config.set("server", "federation", "off") config.set("server", "test", "") config.add_section("authentication") config.set("authentication", "method", "pam") diff --git a/src/wok/control/peers.py b/src/wok/control/peers.py new file mode 100644 index 0000000..2a14815 --- /dev/null +++ b/src/wok/control/peers.py @@ -0,0 +1,30 @@ +# +# Project Wok +# +# Copyright IBM Corp, 2017 +# +# Code derived from Kimchi Project +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-130= 1 USA + +from wok.control.base import SimpleCollection +from wok.control.utils import UrlSubNode + + +@UrlSubNode("peers", True) +class Peers(SimpleCollection): + def __init__(self, model): + super(Peers, self).__init__(model) + self.admin_methods =3D ['GET'] diff --git a/src/wok/model/config.py b/src/wok/model/config.py index b69f2dd..887ceba 100644 --- a/src/wok/model/config.py +++ b/src/wok/model/config.py @@ -34,6 +34,7 @@ class ConfigModel(object): 'websockets_port': config.get('server', 'websockets_port'), 'auth': config.get('authentication', 'method'), 'server_root': config.get('server', 'server_root'), + 'federation': config.get('server', 'federation'), 'version': get_version()} =20 def reload(self, name): diff --git a/src/wok/model/peers.py b/src/wok/model/peers.py new file mode 100644 index 0000000..02afd59 --- /dev/null +++ b/src/wok/model/peers.py @@ -0,0 +1,74 @@ +# +# Project Wok +# +# Copyright IBM Corp, 2017 +# +# Code derived from Kimchi Project +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-130= 1 USA + +import cherrypy +import re +import socket + +from wok.config import config +from wok.utils import run_command, wok_log + + +class PeersModel(object): + def __init__(self, **kargs): + # check federation feature is enabled on Wok server + if not config.get('server', 'federation') =3D=3D 'off': + return + + # register server on openslp + hostname =3D socket.getfqdn() + port =3D config.get("server", "proxy_port") + self.url =3D hostname + ":" + port + + cmd =3D ["slptool", "register", + "service:wokd://%s" % self.url] + out, error, ret =3D run_command(cmd) + if out and len(out) !=3D 0: + wok_log.error("Unable to register server on openSLP." + " Details: %s" % out) + cherrypy.engine.subscribe('exit', self._peer_deregister) + + def _peer_deregister(self): + cmd =3D ["slptool", "deregister", + "service:wokd://%s" % self.url] + out, error, ret =3D run_command(cmd) + if out and len(out) !=3D 0: + wok_log.error("Unable to deregister server on openSLP." + " Details: %s" % out) + + def get_list(self): + # check federation feature is enabled on Wok server + if not config.get('server', 'federation') =3D=3D 'off': + return [] + + cmd =3D ["slptool", "findsrvs", "service:wokd"] + out, error, ret =3D run_command(cmd) + if ret !=3D 0: + return [] + + peers =3D [] + for server in out.strip().split("\n"): + match =3D re.match("service:wokd://(.*?),.*", server) + peer =3D match.group(1) + if peer !=3D self.url: + peers.append("https://" + peer) + + return peers diff --git a/src/wokd.in b/src/wokd.in index 29586a0..dfe85a8 100644 --- a/src/wokd.in +++ b/src/wokd.in @@ -48,6 +48,7 @@ def main(options): websockets_port =3D config.config.get("server", "websockets_port") session_timeout =3D config.config.get("server", "session_timeout") runningEnv =3D config.config.get("server", "environment") + federation =3D config.config.get("server", "federation") server_root =3D config.config.get("server", "server_root") logDir =3D config.config.get("logging", "log_dir") logLevel =3D config.config.get("logging", "log_level") @@ -71,6 +72,10 @@ def main(options): help=3D"Running environment of wok server") parser.add_option('--server_root', type=3D"string", default=3Dserver_r= oot, help=3D"Relative path to server") + parser.add_option('--federation', default=3Dfederation, + help=3D"Register and discover Wok peers in the same " + "network using openSLP. Check README-federation= for" + " more details.") parser.add_option('--test', action=3D'store_true', help=3D"Run server in mock model") (options, args) =3D parser.parse_args() diff --git a/tests/test_api.py b/tests/test_api.py index 6fbee75..f978c93 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -49,11 +49,15 @@ class APITests(unittest.TestCase): def setUp(self): self.request =3D partial(utils.request) =20 + def test_peers(self): + resp =3D self.request('/peers').read() + self.assertEquals([], json.loads(resp)) + def test_config(self): resp =3D self.request('/config').read() conf =3D json.loads(resp) keys =3D ["auth", "proxy_port", "websockets_port", "version", - "server_root"] + "server_root", "federation"] self.assertEquals(sorted(keys), sorted(conf.keys())) =20 def test_config_plugins(self): diff --git a/tests/test_config_model.py b/tests/test_config_model.py index f8b0848..565b4df 100644 --- a/tests/test_config_model.py +++ b/tests/test_config_model.py @@ -30,7 +30,7 @@ class ConfigModelTests(unittest.TestCase): config =3D inst.config_lookup('') self.assertItemsEqual( ['proxy_port', 'websockets_port', 'auth', - 'server_root', 'version'], + 'server_root', 'version', 'federation'], config.keys() ) =20 diff --git a/tests/test_server_root.py b/tests/test_server_root.py index e95a13b..89f34d3 100644 --- a/tests/test_server_root.py +++ b/tests/test_server_root.py @@ -1,7 +1,7 @@ # # Project Wok # -# Copyright IBM Corp, 2016 +# Copyright IBM Corp, 2016-2017 # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -49,7 +49,7 @@ class ServerRootTests(unittest.TestCase): # check if server_root in config is the same used to start server resp =3D request(server_root + '/config').read() conf =3D json.loads(resp) - self.assertEquals(len(conf), 5) + self.assertEquals(len(conf), 6) =20 def test_development_env(self): """ @@ -61,4 +61,4 @@ class ServerRootTests(unittest.TestCase): # check if server_root in config is the same used to start server resp =3D request(server_root + '/config').read() conf =3D json.loads(resp) - self.assertEquals(len(conf), 5) + self.assertEquals(len(conf), 6) diff --git a/ui/js/src/wok.api.js b/ui/js/src/wok.api.js index 06b97aa..4e68eee 100644 --- a/ui/js/src/wok.api.js +++ b/ui/js/src/wok.api.js @@ -71,6 +71,20 @@ var wok =3D { }); }, =20 + getPeers: function(suc, err) { + wok.requestJSON({ + url: 'peers', + type: 'GET', + contentType: 'application/json', + dataType: 'json', + resend: true, + success: suc, + error: err ? err : function(data) { + wok.message.error(data.responseJSON.reason); + } + }); + }, + getNotifications: function (suc, err) { wok.requestJSON({ url: 'notifications', diff --git a/ui/js/wok.peers.js b/ui/js/wok.peers.js new file mode 100644 index 0000000..67e58b0 --- /dev/null +++ b/ui/js/wok.peers.js @@ -0,0 +1,75 @@ +/* + * Project Wok + * + * Copyright IBM Corp, 2017 + * + * Code derived from Kimchi Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +wok.initPeers =3D function() { + var peersDatatableTable; + var peers =3D new Array(); + + var peersDatatable =3D function(nwConfigDataSet) { + peersDatatableTable =3D $('#peers-list').DataTable({ + "processing": true, + "data": peers, + "language": { + "emptyTable": i18n['WOKSETT0010M'] + }, + "order": [], + "paging": false, + "dom": '<"row"<"col-sm-12"t>>', + "scrollY": "269px", + "scrollCollapse": true, + "columnDefs": [{ + "targets": 0, + "searchable": false, + "orderable": false, + "width": "100%", + "className": "tabular-data", + "render": function(data, type, full, meta) { + return '' += data + ''; + } + }], + "initComplete": function(settings, json) { + $('#peers-content-area > .wok-mask').addClass('hidden'); + } + }); + }; + + var getPeers =3D function() { + wok.getPeers(function(result) { + peers.length =3D 0; + for (var i =3D 0; i < result.length; i++) { + var tempArr =3D []; + tempArr.push(result[i]); + peers.push(tempArr); + } + peersDatatable(peers); + }, function(err) { + wok.message.error(err.responseJSON.reason, '#peers-alert-conta= iner', true); + }); + }; + getPeers(); + +} + +if (wok.config.federation =3D=3D 'on') { + $("#peers-accordion").removeClass('hidden'); + wok.initPeers(); +} else { + $("#peers-accordion").addClass('hidden'); +} diff --git a/ui/pages/tabs/settings.html.tmpl b/ui/pages/tabs/settings.html= .tmpl index ccc5b01..5f52839 100644 --- a/ui/pages/tabs/settings.html.tmpl +++ b/ui/pages/tabs/settings.html.tmpl @@ -27,6 +27,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston= , MA 02110-1301 USA