An experiment about personalized front-end of bugs.debian.org¶ ↑
- subtitle
-
E-mail archives + UDD + Simple web front-end = š¤
- author
-
Kentaro Hayashi
- institution
-
ClearCode Inc.
- content-source
-
DebConf20 online August 25, 2020
- allotted-time
-
15m
- theme
-
.
NOTE: Presentation slide is published¶ ↑
-
This presentation slide is available via Rabbit Slide Show
-
slide.rabbit-shocker.org/authors/kenhys/debconf2020-online/ āAn experiment about personalized front-end of bugs.debian.orgā
Personal profile¶ ↑
-
Debian Maintainer (@kenhys)
-
Trackpoint(soft dome) and Wasa beef(Yamayoshi Wasabi Potato Chips) fan
-
Working for ClearCode Inc.
ClearCode Inc.¶ ↑
-
Free software is important in ClearCode Inc.
-
We develop/support software with our free software development experiences
-
We feedback our business experiences to free software
Not talk about¶ ↑
-
Improving bugs.debian.org (debbugs) itself
-
Explaining bugs.debian.org (debbugs) internal in details
Agenda¶ ↑
-
Why so curious about bugs.d.o?
-
The troublesome cases about bugs.d.o
-
How to solve this situation?
-
Demonstrate a front-end
-
Conclusion
We use bugs.d.o¶ ↑
Usage of bugs.debian.org - prerequisite¶ ↑
-
Communicate each other by E-mail
-
Change the bug status by sending E-mail
-
Send E-mail to control@bugs.d.o
-
Use control command in E-mail reply
Why so curious about bugs.d.o?¶ ↑
-
I've received deprecated notification bugs
-
Bug# 956770 growl-for-linux: Depends on deprecated libappindicator
-
Bug# 955899 growl-for-linux: Depends on deprecated dbus-glib
How did you fixed?¶ ↑
-
Bug# 956770 growl-for-linux: Depends on deprecated libappindicator
-
Migrated to libayatana-indicator
-
It is easy because there are enough compatibility
-
I've fixed pkg-config target and include path
-
How did you fixed? (again)¶ ↑
-
Bug# 955899 growl-for-linux: Depends on deprecated dbus-glib
-
Migrated to GDBus
-
Use d-feet to detect interface change (Rhythmbox plugin)
-
Fixed a interface to follow GDBus API
-
-
Send PR to upstream github.com/mattn/growl-for-linux/
-
Fixed a potential Rhythmbox related bug, too
-
growl-for-linux was fixed¶ ↑
-
Butā¦What about other packages? š¤
What about other packages?¶ ↑
-
libappindicator: deprecated in Debian; AppIndicator based applications, please switch to Ayatana (App)Indicator(s)
-
libdbus-glib-1-dev: is deprecated
libappindicator: deprecated in Debian¶ ↑
-
Many blocked bugs
libdbus-glib-1-dev: is deprecated in Debian¶ ↑
-
Too many blocked bugs (Need to scroll!) šµ
How to track many blocked?¶ ↑
-
In this case, UserTags is recommended
Does it work with libappindicator?¶ ↑
-
UserTag is: ayatana-appindicator
Does it work with libdbus-glib1-dev?¶ ↑
-
It is a case that UserTag is not available
-
It should be taggedā¦
The troublesome cases about bugs.d.o¶ ↑
-
UserTag is not always used
-
It may be difficult to see many blocked bugs
-
How to solve this situation?
Use udd.debian.org!¶ ↑
-
UDD can track usertags
-
UDD can track blocking bugs
In my experience through contribution, I need¶ ↑
-
I need easy to:
-
Track a specified bug (tagged/not tagged)
-
Find a bug that no one working on
-
Send control E-mail
-
Find affected bugs
In my experience through contribution (again)¶ ↑
-
I need easy to:
-
<s>Track a specified bug (tagged/not tagged)</s> Use UDD!
-
Find a bug that no one working on š¤
-
Send control E-mail š¤
-
Find affected bugs š¤
How to solve rest issues?¶ ↑
-
Find a bug that no one working on š¤
-
It's a good attitude to fix a bug
-
Send control E-mail easily š¤
-
It's a good attitude to triage a bug
-
Find affected bugs š¤
-
Remember misery bugs #932855, #965164
Supplementary explanation: Find affected bugs 𤶠↑
-
Sometimes grave/critical bug was found
-
2019, critical, e2fsprogs 1.45.3-1 breaks initramfs-tools-core <=0.133
-
2020, grave, policykit-1: fails to install
What about existing solution?¶ ↑
-
Developer Horizon - A dashboard for developers
-
horizon.debian.net/ Not available
What about existing solution?¶ ↑
-
Fabre - fulltext search service
-
fabre.debian.net Around 2005, Discontinued
-
Undocumented Debian ādebbugs internalā {::note}written in Japanese{:/note}
What about existing solution?¶ ↑
-
Debbugs Enhancement Suite
-
A Chrome extension to enhance using Debbugs
-
Tidy up each bug rendering, it's cool (Thanks @lolamby FYI:)
-
ā¦But It is a bit different what I want
-
What about existing solution?¶ ↑
-
Debian Popularity Contest
-
It provides a method to collect installed packages information
-
Public UDD Mirror
-
It provides a statistical information about packages
Starting personal project¶ ↑
-
Concept
-
Make āunstable lifeā comfortable
-
Fix a bug (by finding a bug that no one working on)
-
Triage a bug (by sending control E-mail easily)
-
Avoid affected bugs (by finding affected important bugs)
-
Starting personal project (again)¶ ↑
-
Concept
-
Make āunstable lifeā comfortable
-
Fix a bug (by finding a bug that no one working on) Count comment in bug#NNNNNN
-
Triage a bug (by sending control E-mail easily) Add mailto: link explicitly
-
Avoid affected bugs (by finding affected important bugs) Alert recent grave bugs
-
How to realize concept?¶ ↑
-
Collect bug information regularly (E-mail archives)
-
Collect Popcon data (Fix a bug with your familiar packages)
-
Collect your installed packages information (popularity-contest)
-
Process collected data and make it accessible!
Collect bug information¶ ↑
-
Get E-mail archive with rsync
-
rsync -av rsync://bugs-mirror.debian.org/bts-spool-db/ {::note}It requires 15GiB!{:/note}
-
Use .log and .summary
Extract .log and .summary¶ ↑
-
Use perl module which is used for bugs.d.o (Debbugs)
-
Debbugs::Log, Debbugs::Status, Debbugs::MIME and so on
-
*.log contains multiple raw E-mail content with control sequence
-
*.summary contains metadata of each bug
How to parse .log¶ ↑
my $log = Debbugs::Log->new(log_name => $path); my @records = $log->read_all_records(); for my $record (@records) { ... my $entity = Debbugs::MIME::parse_to_mime_entity($record); print "From: " . $entity->head->get("From"); print "To: " . $entity->head->get("To"); print "Subject: " . get_field($entity, "Subject"); print "Date: " . $entity->head->get("Date"); }
{: lang=āperlā}
How to parse .summary¶ ↑
Format-Version: 3 Submitter: Nis Martensen <nis.martensen@web.de> Subject: libexo-helpers: starting thunderbird fails for some mailto URIs Found-In: exo/0.12.4-1 Tags: patch Package: libexo-helpers Message-Id: <159439678897.3833.10774950364480956704.reportbug@siamos.nes.net> Severity: normal Date: 1594396982 my $summary = Debbugs::Status::read_bug(summary => $path); print $summary->{"tags"}
{: lang=āperlā}
Collect Popcon data and so on¶ ↑
-
UDD: UltimateDebianDatabase
-
You can access with: psql āhost=udd-mirror.debian.net āuser=udd-mirror udd āpassword
-
Schema is available: udd.debian.org/schema/udd.html
-
Import popcon and package data
-
Collect installed packages information¶ ↑
POPULARITY-CONTEST-0 TIME:1596711793 ID:XXX ARCH:amd64 POPCONVER:1.70 VENDOR:Debian 1596672000 1570622400 libgail-common /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libgail.so 1596672000 1596585600 libavcodec58 /usr/lib/x86_64-linux-gnu/libavcodec.so.58.91.100 <RECENT-CTIME> 1596672000 1586736000 libgoa-1.0-0b /usr/lib/x86_64-linux-gnu/libgoa-1.0.so.0.0.0 ... 1596672000 1586217600 libzvbi0 /usr/lib/x86_64-linux-gnu/libzvbi.so.0.13.2 1596672000 1590624000 udisks2 /usr/share/locale/ja/LC_MESSAGES/udisks2.mo END-POPULARITY-CONTEST-0 TIME:1596711817
-
Just upload popcon file
-
/usr/sbin/popularity-contest āsu-nobody
Process collected data and make it accessible!¶ ↑
-
Polling subscribed debian-bugs-dist@lists.debian.org
-
Importing latest E-mail archives from bugs-mirror.debian.org
-
Merging metadata + Web front-end
Prepared server specs for experiment¶ ↑
-
Start with small VPS instance (about 3.26 USD/mo)
-
1vCPU
-
1GiB memory
-
20GiB disk space
-
Max 100Mbps bandwidth
System diagrams¶ ↑
Named my project as Fabre¶ ↑
cite: en.wikipedia.org/wiki/Jean-Henri_Fabre
Why Fabre?¶ ↑
-
Fabre is āThe father of modern entomologyā - the study of insects(bugs)
-
Just reuse previous (maybe) similar project (fabre.debian.net)
Concept of Fabre (again)¶ ↑
-
Make āunstable lifeā comfortable
-
Fix a bug (by finding a bug that no one working on)
-
Triage a bug (by sending control E-mail easily)
-
Avoid affected bugs (by checking installed packages)
Finding a bug that no one working on¶ ↑
-
Show each number of comments (orange background color means no one working on yet)
Sending control E-mail easily¶ ↑
-
Show each mailto: button to control bugs
Easy to view blocked bugs¶ ↑
-
List blocked bugs appropriately
Finding affected important bugs¶ ↑
How Fabre is good enough to work?¶ ↑
-
Now ready to DEMO
Current status of Fabre¶ ↑
-
Tracked bugs: about 89,000 (UDD: 90,780)
-
{::note}Archived bugs: 720,000 (UDD: 85,0331){:/note}
-
Import bugs: every 1 hour
-
Update affected bugs: every 1 day
-
Disk usage (Database): about 1.5 GiB
Weak point about Fabre¶ ↑
-
Sometimes Killed by OOM, need more memory!
Conclusion¶ ↑
-
There are plenty room of improvements to develop
-
In my experiment, mashup some data sources may improve UX
-
bug report (E-mail archives)
-
UDD metadata (Popcon and package metadata)
-
installed packages list (popularity-contest)
Any questions?¶ ↑
Could you speak slowly if you have a question, please?