Dr. Scholtz is just starting in a position with NIST where she will be
involved in evaluation of interfaces for collaborative systems among
other projects. She is currently a consultant in user interface design
and evaluation and has worked with several collaborative systems, ranging
from large on-line web based products to an interesting main frame product
with distributed users. Previously, she worked as the co-manager of
the human factors group at Intel, concentrating on desktop video
products. These products included ProShare desktop video conferencing and
a one to many ProShare add-on, ProShare Presenter.
Dr. Scholtz has a Ph. D. in computer science from the university of
Nebraska. She was on the faculty of the computer science department
at Portland State University for four years prior to joining Intel.
I am particularly interested in evaluation methods for collaborative
systems. In this respect, I am interested in developing dimensions
of awareness in order to evaluate the suitability of collaborative
products and group tasks.
My work at Intel involved desktop video conferencing products and a
one to many collaborative product. Both of these product types were
synchronous as they involved video, some of the awareness problems
were solved. However, we found that participants still desired
the following awareness information: where in the shared editor others
were working, what pieces of information in the shared were contributed
by other participants. In some instances, participants needed to
know exactly where on the page others were looking - having the page
number alone was not adequate. The multipoint version of the system could only
display the live video of the person currently speaking, the other
participants were displayed as still pictures. There were cases when
people were busy working and the video was hidden. In these instances,
participants were also interested in the question of who was speaking.
In the one to many system, a presentation could be given to others
who would watch from their desktop. Awareness issues here were slightly
different. Presenters wanted to know who was there (handled by an
attendance list) but presenters were also extremely interested in knowing
that everyone was receiving the audio and video as well as any
presentation materials. The same is true of live presentations - most
presenters inquire of the audience if they can see and hear. We
provided both an audio and text backchannel for asking questions.
Names were automatically filled in beside of submitted questions and
requests for the microphone. However, many discussions were held
about allowing anonymous questions to be submitted. This was not done,
but this brings up an interesting question of whether awareness should
replicate current possibilities or add new dimensions to current
situations. If so, to what extent is behavior "controlled" by
the dimensions of awareness present in a situation?
Recent work on evaluating a virtual places system has added to my
concerns about awareness. Are there benefits of letting a person
observe a situation but not participate? Outside of technical benefits
what are the social implications of situations such as this?
I have also been involved in the evaluation of a main frame product with workers sitting at distributed terminals. They have expressed desires to know if collaborators are on-line, thereby increasing the chance that the information they have just submitted on-line will be observed soon.,br>
Although, I had originally been thinking that perhaps there were
factors of awareness, I am now beginning to suspect that a high to low
scale of awareness would be useful, with "factors" actually being points
on the scale. For example, a low level of awareness might just be that
others are there, increasing to the number of others present, increasing
to the actual identification of who is there. Adding on factors of who
is doing what with what artifact would add increasing points on the high
end of the awareness scale. Two other scales would be useful in
evaluating the control of awareness, one from the point of view of
what access others had to information about you. The second scale would
be what amount of control you had over what others could see and do.
I indicate these as separate scales because an application could give
others the ability to see everything you do, but you could be allowed
the ability to control the amount of information revealed.
My intention in starting the collaborative evaluation project next month is to look at proposals for evaluating awareness, along with other dimensions of collaborative systems. An interesting question would be where along the suggested scales would a collaborative system have to be in order to be considered "as good as" face to face methods? Would moving up or down on the scale be considered an advantage over current face to face methods? This has been shown to be true for brainstorming techniques where anonymity improves quality of the brainstorming output. By using scenarios and different collaborative products, I intend to do some experiments to look at the effect of different degrees of awareness based on various group activities. I am hoping that the strategies for evaluation developed in the workshop can be integrated into the evaluation work we have planned.