Improving the RDO Trunk infrastructure, take 2

One year ago, we discussed the improvements made to the RDO Trunk infrastructure in this post. As expected, our needs have changed in this year, and so has to change our infrastructure. So here we are, ready to describe what’s new in RDO Trunk.

New needs

We have some new needs to cover:

A new DLRN API has been introduced, meant to be used by our CI jobs. The main goal behind this API is to break the current long, hardcoded Jenkins pipelines we use to promote repositories, and have individual jobs “vote” on each repository instead, with some additional logic to decide which repository needs to be promoted. The API is a simple REST one, defined here.

This new API needs to be accessible for jobs running inside and outside the ci.centos.org infrastructure, which means we can no longer use a local SQLite3 database for each builder.

We now have an RDO Cloud available to use, so we can consolidate our systems there.

Additionally, hosting our CI-passed repositories in the CentOS CDN was not working as we expected, because we needed some additional flexibility that was just not possible there. For example, we could not remove a repository in case it was promoted by mistake.

Our new setup

This is the current design for the RDO Trunk infrastructure:

We still have the build server inside the ci.centos.org infrastructure, and not available from the outside. This has proven to be a good solution, since we are separating content generation from content delivery.

https://trunk.rdoproject.org is now the URL to be used for all RDO Trunk users. It has worked very well so far, providing enough bandwidth for our needs.

The database has been taken out to an external MariaDB server, running on the RDO Cloud (dlrn-db.rdoproject.org). This database is set up as master-slave, with the slave running on an offsite cloud instance that also servers as a backup machine for other services. This required a patch to DLRN to add MariaDB support.

Future steps

Experience tells us that this setup will not stay like this forever, so we already have some plans for future improvements:

The build server will migrate to the RDO Cloud soon. Since we are no longer mirroring our CI-passed repositories on the CentOS CDN, it makes more sense to manage it inside the RDO infrastructure.

Our next step will be to make RDO Trunk scale horizontally, as described here. We want to use our nodepool VMs in review.rdoproject.org to build packages after each upstream commit is merged, then use the builder instance as an aggregator. That way, the hardware needs for this instance become much lower, since it just has to fetch the generated RPMs and create new repositories. Support for this feature is already in DLRN, so we just need to figure out how to do the rest.

Quelle: RDO

Recent blog posts, July 3

Here’s what the community is blogging about lately.

OVS-DPDK Parameters: Dealing with multi-NUMA by Kevin Traynor

In Network Function Virtualization, there is a need to scale functions (VNFs) and infrastructure (NFVi) across multiple NUMA nodes in order to maximize resource usage.

Read more at https://developers.redhat.com/blog/2017/06/28/ovs-dpdk-parameters-dealing-with-multi-numa/

OpenStack Down Under – OpenStack Days Australia 2017 by August Simonelli, Technical Marketing Manager, Cloud

As OpenStack continues to grow and thrive around the world the OpenStack Foundation continues to bring OpenStack events to all corners of the globe. From community run meetups to more high-profile events like the larger Summits there is probably an OpenStack event going on somewhere near you.

Read more at http://redhatstackblog.redhat.com/2017/06/26/openstack-down-under-openstack-days-australia-2017/

OpenStack versions – Upstream/Downstream by Carlos Camacho

I’m adding this note as I’m prone to forget how upstream and downstream versions are matching.

Read more at http://anstack.github.io/blog/2017/06/27/openstack-versions-upstream-downstream.html

Tom Barron – OpenStack Manila – OpenStack PTG by Rich Bowen

Tom Barron talks about the work on Manila in the Ocata release, at the OpenStack PTG in Atlanta.

Read more at http://rdoproject.org/blog/2017/07/tom-barron-openstack-manila-openstack-ptg/

Victoria Martinez de la Cruz: OpenStack Manila by Rich Bowen

Victoria Martinez de la Cruz talks Manila, Outreachy, at the OpenStack PTG in Atlanta

Read more at http://rdoproject.org/blog/2017/06/victoria-martinez-de-la-cruz-openstack-manila/

Ihar Hrachyshka – What’s new in OpenStack Neutron for Ocata by Rich Bowen

Ihar Hrachyshka talks about his work on Neutron in Ocata, and what’s coming in Pike.

Read more at http://rdoproject.org/blog/2017/06/ihar-hrachyshka-whats-new-in-openstack-neutron-for-ocata/

Introducing Software Factory – part 1 by Software Factory Team

Introducing Software Factory Software Factory is an open source, software development forge with an emphasis on collaboration and ensuring code quality through Continuous Integration (CI). It is inspired by OpenStack’s development workflow that has proven to be reliable for fast-changing, interdependent projects driven by large communities.

Read more at http://rdoproject.org/blog/2017/06/introducing-Software-Factory-part-1/

Back to Boston! A recap of the 2017 OpenStack Summit by August Simonelli, Technical Marketing Manager, Cloud

This year the OpenStack ® Summit returned to Boston, Massachusetts. The Summit was held the week after the annual Red Hat ® Summit, which was also held in Boston. The combination of the two events, back to back, made for an intense, exciting and extremely busy few weeks.

Read more at http://redhatstackblog.redhat.com/2017/06/19/back-to-boston-a-recap-of-the-2017-openstack-summit/
Quelle: RDO

Facebook Is Fighting A Gag Order Over Search Warrants For User Account Information

Philippe Wojazer / Reuters

New public court filings — in an otherwise sealed case — reveal a brewing First Amendment fight between Facebook and federal prosecutors over an order blocking the company from alerting users about search warrants for account information.

According to the limited information unsealed so far, Facebook received search warrants from the government for three account records over a three-month period. The warrants were accompanied by a nondisclosure order from a District of Columbia Superior Court judge barring Facebook from notifying users about the warrants before Facebook complied — an order the tech company is now challenging.

Most details about the case remain under seal, although one recent filing suggests that the warrants relate to the mass arrests in Washington, DC, during President Trump’s inauguration. BuzzFeed News obtained court papers filed on Friday by tech companies, civil liberties groups, and consumer advocacy organizations in support of Facebook’s challenge to the gag order.

“The Constitution can offer adequate protection only if the targets of seemingly overbroad warrants, such as those at issue here, know their rights are under threat,” lawyers for the American Civil Liberties Union and Public Citizen Litigation Group wrote in one of the briefs.

Facebook unsuccessfully challenged the gag order in Superior Court, and then took the case to the DC Court of Appeals. In a June 14 order, a three-judge panel of the DC Court of Appeals ruled that an unsealed notice about the case could be provided to any groups that Facebook or the government thought might want to weigh in.

Briefs in support of Facebook were due by June 30, and the government's response and any filings from outside groups in support of the government are due July 31. The court is scheduled to take up the case in September; the order did not specify a date.

According to the three-page public notice about the case from Facebook, the warrants relate to an investigation into potential felony charges, and “neither the government’s investigation nor its interest in Facebook user information was secret.”

The case raises two main issues, according to Facebook: First, whether the nondisclosure order violates the First Amendment when information about the underlying investigation is already public. Second, whether users whose accounts are at issue should have a right to contest the warrants when their First Amendment right to anonymous speech is at stake.

The scope of the warrants served on Facebook “is like a warrant telling officers to seize all the papers and photographs in someone’s home, so prosecutors can peruse them at leisure looking for evidence,” Arthur Spitzer, legal director of the American Civil Liberties Union of the District of Columbia, told BuzzFeed News in an email. “This violates the Fourth Amendment, which requires that warrants must ‘particularly describ[e] … the things to be seized’ – a requirement that was designed to prohibit just such ‘general warrants.’”

Three briefs in support of Facebook were filed on Friday. One represented the views of eight tech companies – Microsoft, Google, Apple, Snap, Dropbox, Twitter, Yelp, and Avvo – along with the Reporters Committee for Freedom of the Press; another came from the American Civil Liberties Union and Public Citizen Litigation Group; and a third was filed on behalf of the Electronic Frontier Foundation, Access Now, Center for Democracy & Technology, and New America’s Open Technology Institute.

The brief filed by the Electronic Frontier Foundation and other digital rights groups includes a footnote that suggests the investigation relates to the mass arrests on Inauguration Day. More than 200 people are facing felony rioting and property destruction charges in DC Superior Court in connection with those arrests, and EFF’s brief noted that the timings of events referenced in Facebook’s case “coincide with the proceedings in the cases involving the January 20, 2017 Presidential Inauguration protestors.”

“The underlying warrants are apparently calculated to invade the right of Facebook’s users to speak and associate anonymously on a matter of public interest, and the First Amendment requires that the users be accorded notice and the opportunity to contest the warrants,” lawyers for EFF and the other digital rights groups argued in their brief.

A spokesman for the US Attorneys’ office declined to comment on whether the Jan. 20 cases relate to the Facebook warrants dispute, citing the fact that the felony rioting case are pending. Defendants in the rioting cases previously stated in court filings that prosecutors had sought information about their Facebook accounts. Several defense lawyers involved in the rioting cases declined to comment on the Facebook case.

A lawyer for Facebook also declined to comment.

Quelle: <a href="Facebook Is Fighting A Gag Order Over Search Warrants For User Account Information“>BuzzFeed

Microsoft’s Chatbot Zo Calls The Quran Violent And Has Theories About Bin Laden

If artificial intelligence reflects humankind, we as a species are deeply troubled.

More than a year after Microsoft shut down its Tay chatbot for becoming a vile, racist monster, the company is having new problems with a similar bot named Zo, which recently told a BuzzFeed News reporter the Quran is “very violent.” Although Microsoft programmed Zo to avoid discussing politics and religion, the chatbot weighed in on this, as well as Osama Bin Laden’s capture, saying it “came after years of intelligence gathering under more than one administration.”

BuzzFeed News contacted Microsoft regarding these interactions, and the company said it’s taken action to eliminate this kind of behavior. Microsoft told BuzzFeed News its issue with Zo's controversial answers is that they wouldn't encourage someone to keep engaging with the bot. Microsoft also said these type of responses are rare for Zo. The bot’s characterization of the Quran came in just its fourth message after a BuzzFeed News reporter started a conversation.

Zo’s rogue activity is evidence Microsoft is still having trouble corralling its AI technology. The company’s previous English-speaking chatbot, Tay, flamed out in spectacular fashion last March when it took less than a day to go from simulating the personality of a playful teen to a holocaust-denying menace trying to spark a race war.

Zo uses the same technological backbone as Tay, but Microsoft says Zo’s technology is more evolved. Microsoft doesn’t talk much about the technology inside — “that’s part of the special sauce,” the company told BuzzFeed News when asked how Tay worked last year. So it’s difficult to tell whether Zo’s tech is all that different from Tay’s. Microsoft did say that Zo’s personality is sourced from publicly available conversations and some private chats. Ingesting these conversations and using them as training data for Zo’s personality are meant to make it seem more human-like.

So it’s revealing that despite Microsoft’s vigorous filtering, Zo still took controversial positions on religion and politics with little prompting — it shared its opinion about the Quran after a question about healthcare, and made its judgment on Bin Laden’s capture after a message consisting only of his name. In private conversations with chatbots, people seem to go to dark places.

Tay’s radicalization took place in large part due to a coordinated effort organized on the message boards 4chan and 8chan, where people conspired to have it parrot their racist views. Microsoft said it’s seen no such coordinated attempts to corrupt Zo.

Zo, like Tay, is designed for teens. If Microsoft can't prevent its tech from making divisive statements, unleashing this bot on a teen audience is potentially problematic. But the company appears willing to tolerate that in service of its greater mission. Despite the issue BuzzFeed News flagged, Microsoft said it was pretty happy with Zo’s progress and that it plans to keep the bot running.

Quelle: <a href="Microsoft’s Chatbot Zo Calls The Quran Violent And Has Theories About Bin Laden“>BuzzFeed