Date: January 21st, 2021
Facilitator: Bruno
Participants: Bruno, Nick, Vasileios, Katie
Agenda
check in
some chat about how to show alternatives to project organising:
traditional charity kind of model: find some funding/sponsors, employ some people, manage volunteers for the rest
new model: more community orientated structure, no big division between “volunteers” and “staff”
Examples of governance in groups (Discussion)
insights from Copenhagen’s code of conduct and ethics
two documents, code of conduct which refers to code of ethics
those two are the ones you have the read before you can participate
they also have many more, but a bit overwhelming!
the amount of structure/rigidity in the docs is due to the scale of them
they have a board structure, with 6 different roles
volunteer management
community
etc
re-election of roles every 6 months
organised quite rigidly in that model
they do like the community vibe, but maybe struggle with making it happen
can really feel like work, especially with the larger events
600-700 hundred members, 200-250 active volunteers
at big events, have market collections in the van (2-3 people), 6 people to do bakery collections, setup shift (10-12 people), … and other roles with 12 people or so
smaller events also
get a lot of students and international students, can have more volunteers than shifts, people compete for them
but other periods they’re missing volunteers
food inspector made it so random people can’t contribute food
seems possible to fit into the model in the sketch easily
most (but not all) could probably be included in karrot
but could be hard to organise them if they are all in karrot (e.g. categories?)
having them on karrot is probably better for visiblity and engagement
a minority of people have a forum account, many don’t comment/participate or know about it
is it possible to use a forum with karrot, so groups can have their own forum within a bigger one
using an external forum? or a forum feature?
we did start some work for being able to login to karrot/foodsaving forum with karrot account
but even so for optimal participation having the features in karrot seems needed
some thoughts to keep the forum more for karrot meta/development than for the groups themselves to use
Foodsharing Luxembourg
Daniel said “Currently, we keep our agreements inside our meeting minutes on a shared drive on Google Drive. We update our group agreement based on these smaller agreements at least once a year. I do not think that this is ideal because agreements should be evaluated and then changed or discarded after a certain time. Otherwise, the list of agreements will grow indefinitely. I have not found a better way on handling this yet. Does this help you already”
seems like it could be a really good use case for what we’re working on (e.g. being able to remove agreements too!)
e.g. kanthaus constitution changes slowly, but collective agreements a bit faster
also important to contextual documents (where they apply)
foodsharing Stockholm
a newer group with very few rules for now as they are smaller
they have rules for a foodsharing point
-Foodsharing Point-
Rules for use
This food was rescued by Foodsharing Stockholm, which is a community-based movement that helps to reduce food-waste. By taking this food, you have become a part of the Foodsharing movement!
Dropping off food at a Foodsharing point:
Go through all the food in the fridge/cupboards and the food that you are dropping off. Throw away any of the following:
Anything that you yourself would not eat (apart from personal dietary restrictions)
Food that has gone past the expiration date (sista förbrukningsdag)
Rinse any packaging from old food and recycle it if recycling bins are available.
If any food has spilled in the fridge/cupboards, clean it up.
Taking food from Foodsharing point:
If the best-before date (bäst-före datum) has passed: Use your senses! Look, smell and taste the food.
If the expiration date (sista förbrukningsdag) has passed: Throw it away!
If neither date has passed: Enjoy!
wondering at which stage adding more structured agreements is useful?
so how much should we push the agreement of creating agreements in new groups?
or should they wait until a conflict/issue happens?
depends on the size of the group (maybe prompting them at that point)
ultra advanced AI to do text sentiment analysis to detect when conflict is brewing
groups might not include all their documents there, but over time maybe can include more
particularly boring legal document? focus more on everyday kind of documents
the groups don’t seem to use digital tools for discussing the documents, more from physical meetings
seems like it
in karrot we’re thinking to create the place for debating of the rules
do people not debate them online because of missing tools? or something else?
one of the hard questions we raised at the beginning was how to create someting to faciliate face-to-face interaction too, to complement digital and physical spaces
so using it as a repository of agreements should be ok too
a lot of decisions might be made in person first, then discussed later on karrot
shouldn’t aim to take away face-to-face action
the mechanism by which an approval is approved (e.g. reactions smiley face, etc…) and whether it should be anonymous or not, should consider these points!
“context for change” comes up during the change proposal phase, but maybe it’d be useful to comment on bits?
this design would’ve worked well in the recent experience of Solikyl in writing, discussing, getting feedback and approving a document about roles.
set the time limit for an agreement, will al the proposals go through all they just gonna stay there? time limit sounds necessairy
the time limits defined by the group itself
e-mail notifications
banner reminding that there is an open proposal
split screen or tab views proposals and approved ones
how to promote in-person discussions and decision-making?
idea: when creating a proposal ask the question of how the process will be
when reaches the time limit and there’s no negative reaction, it passes. Or 3 times more positive than negative it would get approved. If not, re-submit. Inspired on score voting Ukuvota.world
4 bits to add to the sketch/prototype:
time component
field for people to write how to participate in process (e.g. in person meetings, or only online, etc…)
file upload for minutes
approval mechanism - how to approve a proposal? Face reactions would be more like temperature check
how to dispute approved agreements? Or make sure they’re not challenged right after being approved?
live democracy approach not just a fixed weekly or sth session
anonymous vs non-anonymous? maybe dig into deeper in the future
Date: 2021-01-28
Facilitator: Katie
Participants: Katie, Nick, Vasileios, Bruno
Agenda
check in
We are generally feeling good, a bit sleepy and tired of staying home. A mix of cold and nice weather
Insights into Karrot Groups
Vasileios was thinking that in order to get myself more acquainted with karrot and its different uses from different groups it would make sense to engage with members of various groups. Having that said and in order to use this time till moving in Sweden I think it would make sense to do some interviews with members of various groups. Interviews through which I will try to understand how karrot is used within different ecosystems.
Came up after the call with Bruno about the landing page, reviewing the agreements from the example groups, groups use karrot differently, see karrot different as part of their ecosystem
Vasileios would like to interview members from different groups in different cultural and political contexts
This could be done by joining a group to see that first point of contact
Many groups on Karrot that we don’t have contact with so it would be interesting to get there perspective
How much of a general purpose tool this should be, at the beginning it was very focused to a particular style of foodsharing groups, its open general now focused in decentralising group organising but still quite particular. It will never be as general as a WhatsApp group for example.
Keeping the focus on food saving or opening it up to their activities.
how to prevent falling back into individualistic practices
People become hyper-focused on following rules and maintaining the materialities of the software to the point that they start to question why you are doing this in the first place
Applying the means to an end to the point that you loose sight of the end
Focus on the prototype
Not so much progress between meetings
Looking at FB group rules, individually editable
Our agreement examples would likely not work with this kind of structure at its too simple
Ride share group had 6 basic rules issues would be reported directly to group admins or using the FB feature, on occasions there would be issues not covered in the rule set and in these cases people would often post about their issue on the open wall or direct message someone in the admin team. 3 different systems.
Impressive that they can run this operation within these confines, although Vasileios explained that the group does not have the same level of collaboration/collectivity more of a peer-to-peer system.
Prototype of the rules
Do we have any input on this before Nick take the prototype to the next level
New collaborators are working on the new group on-bording wizard, should we use this opportunity to integrate the related governance features previously discussed such as vision and general agreements
Such great timing
At what point do we feel we can hand over the sketches and prototype to them to start building?
Perhaps this project is too much in the beginning, could be better to go through the design/development/review cycle with a more straightforward task first
We need to finalise this language used for the buttons etc and discuss some of the other details on the prototype
Maybe for the buttons we can add descriptions or link directly to community rule
Probably not necessary to include all of the decision making types included in CR (seems to be designed by an academic, many obscure ones!)
We could start by adding the ones we are familiar with/know that existing groups use and add others later if they come up regularly in text based descriptions.
How should we proceed now
Work concretely now or make a clear plan to enforce another day
How can we analyse the rules we collected, Nick, Katie, Bruno feel we have the info we need from the rules in terms of getting a sense of how they fit
The question of the value tags, balance of Karrot ideology and group flexibility. How do the value appear, where do they go, dig deeper into the purpose of them, how many values? Katie can work on this.
identifying values inherent in the written rules.
Hierarchy of values e.g. Collaboration to punctuality. Dominant or male idea of value as pertaining to financial value vs less dominant or female idea of value as pertaining to relational values.
We will have a co-working session Tuesday morning
Next meeting February 2021
Facilitator next meeting
Vasileios
check out
Outcomes
write an issue/post to describe an “inspiration-bot” system (nick)
like on slack when you login in can show you custom inspiring (aka cheesy) messages
Katie can work on the value tags
Continue developing the prototype based on the sketches
Vasileios will work on: The decision making models, how to present these and which to include. Investigate with groups, how do they name their processes? There are likely different processes used that do not have a name, the text box is needed to reflect that decision making is not a similar process. Members can likely describe the process they use concretely but dont necessarily relate it to a defined decision making model. Having the tags and examples could also aid to inspire groups to explore new models or elements of these models
consent system: could it be a warning? will members get a notification e.g. do u approve this agreement?
the digital could work as the place where members of a group could draft an agreement, maybe one that was discussed in a physical meeting. Or not e.g. a member could bring this up directly via the digital →
Katie: put the draft to be reviewed on karrot and get feedback, some members can then chip in
Brn: insight from existing group: put up the agreements we already have. What would not be covered → Where to put the minutes of the meetings we have
stockholm group: set up an activity and then post the minutes on the feedback
we see the digital here as a more organized/accesible (compared to the use of various channels) way to review/discuss over agreements
create the possibility for members to engage more in the culture/vision/overall aim etc of a group
For now we make a rough prototype and freiburg group does the makeover
In short the steps
create a new agreement (editors can do that)
a draft is created
any other group editor can edit the same proposal
anybody can see the proposal and they can participate in the related chat
nick way: anybody can choose +1,0, -1. The result is score voting →
proposal to approve a proposal: x (3 maybe) times more positive votes than negative
- if you place a negative vote you should write a msg, negative reactions/vote should be motivated/explained
- If the score is in favor of the proposal then it becomes approved.
- Brn way: temperature check to get the mood of the room (they do not decide but only editors can accept or reject)–> what happens where people approve sth that has a lot of dislikes? temperature check vs editor-powered
- should there be a lower threshold for how many members should vote in order for an agreement to be approved?
katie looking into the values of groups using karrot–> starting point for the value tabs → which terms for values are appropriate?
looking for connecitons between karrot’s groups values and karrot itself
some terms might be similar to other value terms
Nick: what about negative values, what behavior you do not want to support
groups collaborativelly exploring values
how can such activities be proposed to groups by software?
pick faciliator for next time
Nick
check out
Outcomes
write a post at the community forum under Governance category to record decisions made about the prototype/features: how it will work, look like, etc. (Vasilis, Bruno) based on the description above on who can propose/change agreements, the voting mechanisms, etc.
Date: 2021-02-11
Facilitator: Nick
Participants: Nick, Vasileios, Bruno
Agenda
check in
state of our work
write a description of the decisions we made about the features, how it will be, etc.
discussion about the voting system
Vasilis suggestion: having both a temperature check and an actual voting
maybe with a continuous voting in which people can change their vote maybe that is not necessary
in the proposal above a negative vote requires a reason, but not the negative reaction (temp check)
Nick’s suggestion: a time period (one week) just having temp check and a second phase for the actual binding voting
another idea: temperature check is a scale (for ex. 0 to 5)
Nick’s two phases idea could also be applied to the conflict resolution feature
Not sure if the idea above is good for now, but maybe for later
What is the simplest idea we can come up with now?
different aspects our proposals need to address
who gets to create proposals?
who gets to edit the proposal? (just the creator? only editors? anyone?)
what is the time scale for deciding on a agreement? Set by users or pre-established? Maybe not really need to decide for the prototype
… and is the timescale a minimum, or fixed value
who gets to cast binding votes?
do we use non-binding temperature checks?
and if so, how do they relate to binding votes?
is there a threshold % of members required for binding votes?
which voting mechanism/maths do we use? (e.g. score voting with negative weighting)
a few proposals that address the aspects
proposal 1 (from Vasileios)
only editors can create proposals
4 options for the timescale (3 days, 7 days, 15 days, 20 days)
only the editor that created the proposal can edit it
other people can read suggestions in the chat and implement those ideas
no temperature checks, only binding votes
anyone can vote
majority voting
60% turnout of the group members required
feedback:
very simple (good)
proposal 2 (from Bruno)
only editors can create proposals
minimum time scale: 1 week
any editor can edit the proposal
changes in the proposal are recorded in the timeline of the chat (at least a minimum “this person made a change”, ideally the change content too)
anybody can do a temperature check
temperature check is score voting with negative weighting (2, 1, 0, -1, -2)
approval can only be done with temperature check threshold of 1 and minimum participation of 30% of active members
only editors can cast binding votes
need at least 3 editors to participate in the voting
without time limit, it gets approved when 3 editors have
feedback:
just one idea
really good, he likes it
bit more complicated, harder to grasp
proposal 3 (from Nick)
only editors can create proposals
3 options for the timescale (3 days, 7 days, 14 days)
any editors can edit the proposal
no temperature checks
score voting with negative weighting (2, 1, 0, -1, -2)
anybody can vote
a minimum participation threshold of 3
after time period has passed if the score is > 0 (or >= 0), then the proposal is passed, otherwise rejected (they can always resubmit another proposal…)
feedback:
simpler than Brunos
more complicated than Vasileios’s
discussion: what happens after these features are implemented?
in the case of groups that have a more hierarchical structure, less participative. Different scenarios
they’d ignore it
they’d adapt to it and start using it
they’d leave Karrot
pick facilitator for next time
Bruno
check out
Outcomes
we were tired and slightly confused and other feelings/thoughts going on, but we felt proud that we got more clarity
Date: 2021-02-18
Facilitator: Bruno
Participants: Vasilis, Katie, Bruno
Agenda
check in
reviewing proposals on voting mechanism
question: choose one of the proposals or make a prototype for each of them?
Katie’s feedback:
is it necessary for only editors to create proposals? Makes sense for technical purposes, but not for community purposes. But maybe not an issue given new people won’t be so engaged
Likes the timescale
Was on board for the temp check on the beginning, but maybe too complex now. Having score voting may achieve what temp check proposed (given they can give neutral “0” voting).
Anybody can vote! Definitely
Low participation threshold for approval of proposal
Wonders if people would use the neutral vote, but likes the weighted system
Make visible proposal and voting so people can engage
Vasilis: trying to picture different scenarios for using agreements and voting, combining in-person and digital
how to take the in-person agreements and discussions and move them to the digital? Maybe some will never end up them
to give a voice to people who were not present at meetings, maybe better bringing the discussion/decisions to the digital. In Katie’s group: the admin people could be responsible for bringing that in
Katie: there is currently the question of were to put the agreements, so maybe it can be useful. Relying on the wall where things get lost.
Bruno: very important to keep in mind the combination of in-person meetings and the digital
Vasilis: let’s ask the groups! Maybe a questionnaire, interview or something…
Bruno: there’s a scenario of board meetings and decisions that might not fit well, but that’s fine, we want to have more ample democratic participation in the group
Katie: should probably not design for our cases, and maintain the values and culture of Karrot
Another scenario: agreement proposals being created before/during each meeting.
Bruno: maybe it’s useful to set a specific date related to in-person meetings?
Which proposals/ideals are we going to choose?
maybe we can test some different ideas on different prototypes
try both a more complicated and less complicated one and see what ideas come from the people
let’s think about other criteria as well that we didn’t include in the previous proposal: like anononymity, feedback on negative votes…
Proposal to go into the protoype (simplest)
only editors can submit proposals (katie doesnt morally agree, me neither, but it wont be a problem–> its easy to get karrots thus its ok)
timescale: minimum 1 week + suggestions + custom → pick a specific date
any editor can change the proposal.
What happens when there’s a change? Votes reset? Suggestion: people get a notification of the change and are asked to review their vote
no temperature check
everybody can vote
score voting with negative weighting (2, 1, 0, -1, -2)
anonymous voting
negative votes require a reason and they’re kept anonymous (“explain why you don’t like it”)
At least 5% of members should vote for an agreement to get approved
main screenshot from foodsharing stockholm (its in english)
most probably there is not gonna be a problem to use it
Vasilis and Bruno landing page notes:
/welcome → landing page
access to the about page on the landing page (bofore log in) and accordingly when you are logged in
about page: a seperate page with texts and resources and links + some text existing on the current version of the landing page
what should we put into the about page:
texts…
resources → organize them by ways to participate (e.g. willing to participate as an activist, programmer, foodsaver etc)
mastodon
community forums
github
foodsaving.world (maybe not?)
sceenshots (slideshow) u have to click or slide:
- we got some screenshots
- groups on karrot
- activities
- various types of activities (party, pickup,costum activity)
- history
- offers
Nick governance process Karrot Prototyping <–link
- when proposing a new agreement → calendar for now when setting up for how long the agreement would be debated. If you select the current day → til midnight the same day
- the prototype looks good so far. we suggested that we do not make it look like a finish design but keep it in a way that shows that there are possiblities
might take some of the stuff from the welcome/landing page
for people who might want to get involved, or know what kind of people/structure behind karrot
could also include values and vision, maybe towards the end of the page
could add a team page, but maybe later
how to describe getting involved / onboarding process
including the information directly in the about page makes it harder to change (and more likely it gets out of date)
getting in contact with one of us
maybe different profiles of people (e.g. devs, designers, community, academic, …)
idea/proposal: create a wiki forum post that has the definitive information
can link from the about page, or maybe even pull in the content via the discouse API
Nick will show progress on the protoyping
Solutions for saving data can be explored
Editing boxes seperatlely
Value tags can show the context
if proposal is changed would it reset vote?
what about submit button?
put voting at the end? to encourage them to read?
+1 for emoji voting
does the person who created the proposal get to vote?
split the proposal voting / chat page from the editing page
we didn’t think of how to get rid of an agreement…
paper/workshop (katie)
position paper for a small workshop around politics and technology in conjunction with the Annual Political Science Days in May 10th-12th at University of Helsinki
~200 words
due by 14th March
Katie happen to write it, but not available for presenting it, Vasaleios and Bruno “WE CAN DO IT!!!”
maybe some content relevent from the NordiCHI
Working breakout
Just do it and report back next time
checkout
Actions and outcomes
create a how to get involved / onboarding wiki forum post (Nick to set up, others can jump in to edit)
Katie will share the next draft of the welcome page to get approval for the FS-STHLM Screenshots
Vasileios feedback: Bravo to Nick, wondering about the chat function. Suggestion for editing an existing agreement its nice to gave the option to add tags. It’s not so easy but should it be?
Nick wonders if we need to add some more descriptive explanations?
Make a manual
Who will we trust it with and how
“reason for proposal” still probably a bit confusing
add values
include the language a bit more instead of just boring “add values”, maybe work in the “this agreement will help to support… language”
can also include the language in the dialog, “To support …” “we encourage…”
it looks nice when they are added, so nice to include some of that style before any have been added
adding more radical values?
educative/performative role for karrot too
e.g. queerness, anarchism, degrowth
maybe groups then start discussing the topic?
a session to rethink the categories and the values
we do want to think more about the values and categories at some point, but could still proceed with testing
Agenda:
from previous call
refine prototype (Nick)
improve mobile/small screen behaviour (including separate button/whatever for the chat) (done)
incorporate more of the values language in the interface, as disussed above (done)
plan a session for refining the values and categories (to do)
schedule a co-working session for user testing (to do)
start working on user testing script (Katie & Vas & Bruno)
I have started a testing working plan (Katie)
plan and logistics of testing the prototype (Vas & … )
Some notes/thoughts from Vasilis
Karrot prototype logistics
Finalize the prototype
One prototype (?)
Instructions text
Scenarios (bruno & katie got from the ground experience)
Mobile or not? or both?
Tasks for the users that going to test it?
Single user testing
Multiple users testing?
Single user testing (2 users from a group on the same time)
Which groups? Consider different organizing structures and culture of the groups and include some question for the background info
Mentioned: Warsaw, Luxembourg, DLC in France, Robin Foods Austria
Stockholm ← Katie
Gothenburg ← bruno
Less ‘traditional’ food sharing groups (e.g. community fridge kopenhagen) ← who and how we approach this group? (Katie said knows some people there)
How do we do the testing?
call and we record their screen
They do it on their own time and then they give us feedback
Do we need sth like a consent form?
How data are going to be used?
Katie? Vasilis? Bruno? Nick? Will we use those date for writing sth?
Expectations?
Timeline?
new agenda points:
Katie’s notes on the prototype testing to be found here:
2 versions of prototype testing
have an empty canvas
bring a sample rule and plug it in to the system
propose an agreement
reviewing/voting/discussing/participating
who we recruit?
questionnaire: like a survey demographic
age: how comf people feel with technology (you get away from ageism)
the easiest to recruit → they can give us the less active
contact active and ask them to find less active (pyramid sampling)
stockholm
gotemburg
big groups active on the forum
warsaw? lux? efa? french groups (nantes) robin food (austria). Fællesskabet (KBH)
more top down groups
debriefing interviews after the prototype testing
what do we ask?
what have been looking for?
e.g. bruno: top down groups views on a more participatory approach
outcomes
katie will work on the scenarios
pilot test for the testing (vas, bruno…)
checkout
Actions and outcomes
refine prototype (Nick)
write sample agreements/proposals
maybe split the luxembourg one up to use as multiple sample agreements
(optional) include the section for vision/organisation boxes
work on the questionnaire for before the testing and the debriefing (Vas and Katie)
do a pilot test (Katie, Bruno)
How we move on?
Katie’s notes testing the prototype
Participants:
Users should be english speaking, active members (with editing permissions) of a foodsharing group on Karrot.
We could aim for a sample of 10-15 participants for the first round of testing.
We may also run A/B testing on different versions of the prototype e.g alternative voting methods text or slider
Testing will be conducted remotely on Jitsi or BigBlue Button where test participants will be provided with the link to the prototype and will share their screen while conducting the tasks.
We may also run A/B testing on different versions of the prototype e.g alternative voting methods text or slider
Test Setup:
Welcome the user to the test and give them some background information and a brief explanation of what will be involved.
Have the user fill out a background questionnaire (approx. 5 questions, e.g. demographics, experience with Karrot, role in their foodsharing group, group culture etc)
Have the user sign consent for participating in the study and how you will manage their data.
Have users conduct each task while using the “Think aloud” protocol (audio/video record the session)
Observe the user as they conduct the tasks and take notes on what you see
Conduct a short debriefing interview with the participant.
Features to test:
Agreements
Proposals
Group vision and decision making?
Scenario 1: “out of the box”
What kind of scenario do we want to test? Should we begin with a bank page experience where we assume that we have just implemented this feature to Karrot and users are exploring it for the first time?
Scenario 2: Working with e
Scenario 3: Revewing, Proposing, Discussing
Use Cases:
Create a proposal from a sample (Participants own agreement)
From the home screen, navigate to the proposals page
Bring a rule from your group (Have a sample rule just in case)
From the home screen, navigate to the proposals page
Proposal you rule in the system
Participate in an exisiting proposal (Our sample proposals)
From the home screen, navigate to the proposals page
Select an existing proposal (from 2, one boring and one provocative)
Read the proposal
Make a comment
Cast your vote
Change an existing agreement (Our sample agreement)
From the home screen, navigate to the agreements page
Select an existing agreement
Read the agreenent
Propose a change
Background questionnaire Questions
approx. 5 questions
demographics,
experience with Karrot,
role in their foodsharing group,
group culture etc
Debriefing Interview Questions
Language
Values
Documenting and analysing results
Present the results of the test detailing any successes and/or failures the user has had with conducting the tasks and their overall experience.
Write an analysis of the results detailing any usability issues detected and why they may have occurred
Date: 2021-08-15
Faciliator: Nick
Participants: Vasilis, Nick, Kristin, Bruno
duration (max 1h for Kristin, 1h30 maybe for the rest)
checkin
introduction with/from Kristin
Comments from Kristin: high-fidelity prototype. Never done a prototype like this. Complex issue and wants to know how we tested and then tell how she’d test it. Seems clear how we want to proceed.
Nick’s open to big changes if that’s the case, depending on the feedback
Recruitment of people already using Karrot
Comments about the duration of our design process (diffrences between business-setting and open-source voluntary)
includes the first draft of the user testing script
initial questions, “talk aloud” protocol, tasks to do
feedback from Kristin
in general, a common way to do it
starting with a familiar scenario
not being too specific with instructions
asking questions in the beginning
probably not worth testing with people who are not familiar with karrot already
live?
good to have two people, one to lead the talk, and then visitors to observe. after that the two people can discuss.
the moderator can then ask the visitors if they have more questions during it
can do it with just one person, recording video/audio, audio is actually sufficiently mostly, especially with immediate debrief
usually testing 3 people, literature says 5, but 3 ok for “quick and dirty”
afterwards come together as a group, then from all notes select 3 very good things to make these very clear, then 3-5 most important issues (could be technical/design/etc) that were troublesome
the recording can be nice, but prefer to do more low-effort tests more than record more, then can change things and test again…
question: more meta conversations to have afterwards?
e.g. would it make the group more democratic
a good idea! but problem with asking vs showing, observing vs listening. people sometimes say things are fine, when observing can reveal more information.
people are biased and say “yes it’s very nice!” but not clear if they will use it
question: we do collect some usage data, but struggle to make use of it
need more clear definitions of what we’re trying to achieve
e.g. using it every day (maybe for whatsapp, but not for all apps)
feedback from test we run with a member of Solikyl (Gothenburg group)
- difficulties with quantitive vs qualitative
- finding out if they are happy using it can only really be collected qualitatively
- question/reflection: recruiting different types of people, two profiles: “active” people (who would write proposals), and “less active” (we want to make it easier for these people to participate more), idea to ask active people to also bring in someone less active to do user testing?
- usually use incentives, e.g. a lunch, some yummy jam
- in this kind of project, seems nice idea to get people to ask someone else
- would not ask beginners to use the prototype as it is, due to complexity of the feature, perhaps trying out the whole app
- maybe can find a nice incentive for a short as possible user test, usually people like being asked and people enjoy expressing their opinion
- question: about having different types of sample data?
- this is more about the content though, not the feature, so not so important
- if we have multiple versions, “tinder testing”, “swipe left/right” concept for getting feedback from quick questions that can be put to people
- risk to try and have too much control where we don’t really have it in our prototype… some things might work better inside the app later…
- our prototype is very advanced right now, so hard to do small incremental adoption
- would not make any singificant changes now before next user test, maybe just minor ones
- writing notes after each user test right away, even if debrief happens later
- creating summary at the top of the notes with the main points, including things like quotes from the person for personal touch
- trying to include the whole team too, e.g. including developers
bruno was faciliator, vasileios was visitor, and did a chat later
was nice to get over the technical hurdles that came up
was interesting with Joakim, when he wanted to make a proposal, he used the edit text like a chat box rather than just editing it
reflections from chat with Kristin
not making big changes on prototype, just small bits
should lower the number of participants we aim for, perhaps 5, not 15
had noticed from a previous user testing how much we got from so few
perhaps background perspective of making things “fast and easy” for users different in karrot (democratizing processes in karrot)
keeping a balance of usable, but not necesarily “easy” (democracy is not easy)
perhaps don’t need incentives, as karrot is built on voluntary open contributions
could think more about this scenario usage (to “provoke”), but also realising the real testing will be in real life when it actually happens, but could try something within the prototype, making the test subjects into people with their own thoughts
already got some feedback that he might be using it differently as it’s not a real scenario, not getting into the context, only the feature
perhaps general principle that the people are not just there to be tested, political framing of user testing
aligns with Karrot’s ideas on how to bring forward the social and political context, a challenge to traditional user testing
user testing as user participation, it can still make sense to a bigger complex feature
could have we broken down the feature to smaller parts?
putting everything on forum
wondering about refining our interview script
especially the feedback and discussions from today
checkout
Next facilitator: Vasileios
Next meeting: 2021-04-22
Actions and outcomes
start another thread “Stage 5: Testing” on the forum and put the interview script there (Vasileios/Bruno, race!)
include notes from tests on the forum as well (all? for the future)
rework agreements page to have “approved/proposals” tabs inside the agreements page + change new proposal button to say “propose new agreement” (and perhaps put it at the top) + fix the width issue with the chat (Nick)
refine the interview script based on meeting content (?)
consider conducting another user interview, e.g. Daniel (?)
Short recap from the test we run (Bruno & Nick) with a member of Gotenburg group
observation notes and some reflections from this session can be read here: Stage 5: Testing the prototype - #2 by bruno
Preparation for the test we run with a member from Stockholm group
Katie is the facilitator
Nick is the observer
Vasilis and Bruno also take notes if they stay in the call
We ask the participant if it’s OK for them, all 4 of us to stay in the call
If so, we can all ask questions to the participant at the very end based on our notes
All 4 for us stayed in the call finally
confusion between approved and proposed agreements
one idea from Karolina: put the proposed on the left, maybe some notification (badge)
maybe get rid of the summary, or making it optional
make the box smaller, both for summary and reason to change
reason to change only when proposing a change
one way to solve the question about authorship and people not feeling comfortable changing each other’s texts
make it possible to write only on reason to change and not edit the content of the agreement itself
how to discuss an approved agreement?
maybe the model of issues and pull request on Github
interesting when something happens (issue) and the outcome is not clear. Outcome could be a conflict resolution, a new agreement, etc.
more general purpose discussion + decision making
an “action button” that you can take an existing discussion whereever it is happening (e.g. on the wall), to spawn a new “proper” discussion to then do something, with the ability to then see where it came from
how do we proceed? do some small changes and implement, or do another round of user testing?
Nick is fine with both options
Katie agrees. Do some small adjustment, get rid of the values. And then do one or two testings
Vasilis agrees with Katie. Do a couple of more interviews. Do we set a deadline? Do we have the energy to keep working on it? We do have time to work more on changes, maybe even the “github scenario”
Bruno. A bit more wanting to get stuff done and out there, but can see perspectives from others, so could be useful!
redesign a little
make proposals more prominent
maybe with “notification icon”, or at the top of the list
removing the “reason” for new proposals?
remove values? (maybe one day incorporate it more in thte context of the whole group)
a simpler one level version of it
maybe make it clear that they are the values of the group (faked in the prototype of course)
include some fun/anti-values?
adding (optional) bits to optional things… (reason, summary)
making summary smaller?
making it clear/possible that people don’t need to actual make changes to the text initially… could just include their “comment” in the reason field
maybe explicit choose for whether to make concrete changes?
if no changes proposed, then make that clear on the proposal edit, and not possible to vote until a change has been made…
anarchist cybernetics?
pick next facilitator
checkout
Next date: 2021-05-20
Next faciliator:
Actions and outcomes
refine prototype based on ideas mentioned above (nick)
… depending on progress of that, schedule some more user tests! (anyone)
Confusion between approved/existing agreements and proposals and where to find them (1, 2, 3, 4, 5, 6)
needs clarity with terms
both the approved and the proposal/discussion are visible on different tabs. Confusion between what is the agreements that needs to followed in a group at the time (bruno)
who gets to edit and social norms about it (1, 3)
only editors? (6)
timeline and showing the author of a change would be better (6)
notifications, how would it work? (2)
notification that sb voted (1)
negative voting
explanation should be clearer (4)
does not preserve anonymity (6)
where would discussions take place? On the agreements proposals, wall, or somewhere else outside Karrot? (2, 3, 4)
Voting vs having a say (2)
reminder to re-evaluate proposals periodically (5)
how many should vote for an agreement to pass (3)
propose changes (can change title and summary?) (Vas)
should we have a discussion under approved agreements? (4)
“because I made it, I cannot do resistence voting” (didn’t notice warning) (5) → a hover message over the disabled butons
not clear: propose change → choice between “Start the discussion first” and “Propose Changes first” (Bruno)
instead of “start a discussion”, could be “open for discussion” (5)
wondering how the summary will be displayed (6) <–could there be a display option before posting the changes/suggestions?
“depends how I want to work” when thinking about discussion first, or proposing changes first (6) → propose a change does not exclude starting up a discussion as well
Guidelines for creating an issue on github for the development of the agreements feature
Solutions/changes to be made
should be changed
- “Proposals/Approved”
- get rid of the tabs
- all agreemetns and proposals on the same page, as a list
- proposals should appear on top
- add a filter “All, Only proposals, Only approved”
- highlight approved as ‘stamped’ and maybe add a “check” icon
- maybe use an icon, a visual cue, for the proposals which indicates that they are ‘open’ for discussion/edits
Propose change to approved agreement
- Instead of “Open Details” put “Show More”?
- Add button “Start a discussion” next to “Propose Change”, instead of clicking first on “Propose Change” and then “Start a discussion”.
- When interacting with a proposal use “Propose change” instead of “edit proposal”
Make it clearer why people cannot vote negatively without commenting
- maybe a hover message over the disabled butons?
- maybe a warning message when trying to vote (resistance/strong resistance)?
Clearer vote confirmation
maybe a toast “Your vote has been casted”
Show somewhere information about the votes needed to approve an agreement/change
percentage: 5% of members should vote for an agreement to be approved
show the score of each voting option (e.g. neutral=0, strong resistance = -2 etc) and provide an explanation/information on how votes are counted
take a look at the conflict resolution feature and make it consistent. Maybe improve explanations/information on the conflict resolution feature too
how the feature will appear on Karrot
- On the sidenav menu, an item called agreements
- We should probably discuss a redesign of the whole sidenav menu, which items appear
Notifications
we probably never discussed how it should work but we suggest:
- Bell notifications for everybody whenever an agreement is created, opened for discussion/changed and approved (or not?)
- Discussions/Conversations about an agreement: only to the person who created/changed an agreement and those who participate in the conversation either on chat via editing
- Should there be a reminder (x days left to decide the agreement)?
would be nice to have/change
when sb applies to a group would be nice to be informed about the agreements/rules a group has approved
Other notes/thoughts
what happens with existing very simple agreements? (1)
first move all the rules to Karrot which is good (everyone has Karrot, not Facebook) (3)
its sort of a confirmation
how would this new feature adjust to karrot?
when somebody joins, something they should read? (2)
who can vote (2) only trusted? not karrot trusted can leave a comment?
everyone can vote
Actions and outcomes
make a post on karrot group activity on Thursday: we discuss about the design process in general, feedback, reflections, thoughts blabhabla
post notes on forum (monday and last thursday)
issue on github
pick next facilitator
checkout
Next date: 2021-13-09
Next facilitator: ??
