Polyphonic Analysis of Discourse in Texts and
in Collaborative Learning Chats
Ştefan Trăuşan-Matu
University Politehnica ...
Contents
 The Polyphonic Model
 Polyphonic Analysis
 Implementations of the Polyphonic Analysis
 The PolyCAFe Analysis...
Polyphony- An Unitary Model of
Human Communication
 Mediation:
 Using natural language (words) in
 texts
 hypertexts
...
Polyphony- An Unitary Model of
Human Group Communication
 Considering rather a dialogistic, post-
structuralist (Bakhtin...
Polyphony- An Unitary Model of
Group Communication
 Applicable to:
 Small groups (e.g. virtual teams
collaborating by ...
Polyphony- An Unitary Model of
Group Communication and
Intertextuality
 It may be used in IT implementations
using
 Nat...
The Polyphonic Model
7
Stefan Trausan-Matu
12/14/2013
Polyphony
 Appears in music (e.g. J.S.Bach) and in texts (Bakhtin)
 The Polyphonic
 Model (Trausan-Matu, Handbook of Hy...
Polyphony
 A group of participants that, each of them
keeps their individuality, personality, creativity,
but also colla...
The Polyphonic Model
 Polyphony = Model of collaboration and interaction
(Trausan-Matu, Stahl and Zemel, 2005)
 Human c...
11
Stefan Trausan-Matu
12/14/2013
Polyphony
12
Stefan Trausan-Matu
12/14/2013
Dialogism and Polyphony (Bakhtin)
 Mikhail Bakhtin:
• Utterances (not sentences) should be the unit of analysis
• “These ...
Bakhtin’s Polyphony
 Everything is a dialog (applying not only to
speech and text)
 Utterances
 Voices
 Inter-animati...
Utterances
 Utterances (not sentences, as in ‘classical’ linguistics) should
be the unit of analysis (Bakhtin)
 Utteran...
Voices
 Distinctive presences in a group, influencing the
other voices
 Generated by utterances (singular or repeated)
...
Inter-animation patterns
 Longitudinal
 Adjacency pairs
 Repetitions
 Elaboration
 Convergence
 Cumulative talk
 Re...
The Polyphonic Method applications
 Chat conversations with multiple participants for:
 CSCL:
K-12 students solving math...
Computer Support for the
Polyphonic Analysis
19
Stefan Trausan-Matu
12/14/2013
LTfLL - EU FP7 Project (2008-2011) and
NSF Virtual Math Teams Project
http://www.ltfll-project.org/ http://mathforum.org
...
Chat-based CSCL
 K-12 students solving mathematics problems both
individually and collaboratively in the Virtual Math
Te...
Theories for analysing
multi-parties conversation
 Discourse analysis (Tannen)
 Conversation analysis (Sacks, Jefferson,...
Analysis methods
 TF-IDF
 Latent Semantic Analysis
 Naïve Bayes
 Social Network Analysis
 WordNet (wordnet.princeton....
Analyis methods
 TF-IDF
 Latent Semantic Analysis
Almost all are
based also
on
 Naïve Bayes
a two
interlocutors
 Soci...
25
Stefan Trausan-Matu
12/14/2013
26
Stefan Trausan-Matu
12/14/2013
NLP pipe
 spelling correction, stemmer, tokenizer, Named Entity
Recognizer, POS tagger and parser, and NP-chunker.
Stanf...
Social network analysis
 Consider explicit and implicit referencing as arcs
between participants, which are the nodes
 ...
Polyphony, Inter-animation and
Collaboration analysis
 Assign an importance value for each utterance
considering several...
Computational details
(Trausan-Matu, Dascalu and Dessus, ITS 2012;
Dascalu, Trausan-Matu and all, 2010, 2011))
30
Stefan...
Representations:
Conversation graph
 For each participant there is a separate
horizontal line in the representation
 Ea...
Representations:
Weaving of Voices
 Voices in the conversation graph
 Participants = horizontal lines
 Threads of repea...
33
Stefan Trausan-Matu
12/14/2013
34
Stefan Trausan-Matu
12/14/2013
35
Stefan Trausan-Matu
12/14/2013
36
Stefan Trausan-Matu
12/14/2013
37
Stefan Trausan-Matu
12/14/2013
37
38
Stefan Trausan-Matu
12/14/2013
38
39
Stefan Trausan-Matu
12/14/2013
39
40
Stefan Trausan-Matu
12/14/2013
40
41
Stefan Trausan-Matu
12/14/2013
41
Validation
(Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2010;
Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2011)
42
S...
Other applications
43
Stefan Trausan-Matu
12/14/2013
Analysis Dimensions (types of voices)
in Face-to-Face Settings
(Trausan-Matu, in Suthers & all, (eds.) 2013)
 Spoken dial...
Inner utterances, inner speech
 “Mead (1934) called thought a <<conversation
with the generalized other,>> implying that...
Detecting important moments
(Chiru and Trausan-Matu, ITS 2012)
46
Stefan Trausan-Matu
12/14/2013
Graphical representation of topic’s
rhythmicity
(Chiru, Cojocaru, Trausan-Matu and Rebedea, ISMIS 2011)
47
High
rhythmic...
Determination of collaborative
regions
(Banica, Trausan-Matu and Rebedea, CSCL 2011)
48
Stefan Trausan-Matu
12/14/2013
Metacognition
(Trausan-Matu, Dascalu and Dessus, ITS 2012)
 Combining Chats and Texts
 Intertextuality
49
Stefan Traus...
ReaderBench
(Dascalu, Trausan-Matu, Dessus, 2012)
50
Stefan Trausan-Matu
12/14/2013
Creativity fostering
(e.g. brainstorming)(Trausan-Matu, 2011)
51
Stefan Trausan-Matu
12/14/2013
Opinion mining
(Musat, Velcin, Rizoiu, and Trausan-Matu,
2011)
52
Stefan Trausan-Matu
12/14/2013
Topic Modeling
(Musat and Trausan-Matu, 2011)
 No generally accepted definition for a “topic”
 Document clusters
 Abstr...
Topic Modeling
(Musat and Trausan-Matu, 2011)
 LDA/pLSA/hLDA/CTM
 Each newer version corrects some flaws of the earlier
...
Intertextuality analysis
(Ghiban & Trausan-Matu, 2012)
Voice I
Voice I
Voice II In dialog
Voice III
Voice II
Voice III
...
Intertextuality analysis
(Ghiban & Trausan-Matu, 2012)
56
Theme 2 and Theme
3 may have the
Stefan Trausan-Matu
same word...
Analysis of interethnic discourse
(Trausan-Matu, 2012)
 Needed a corpus of texts with time stamps
 Extract recurrent con...
Time Series Analysis of News
(Badea & Trausan-Matu, 2013)
58
Stefan Trausan-Matu
12/14/2013
Music Composition at K-Teams Laboratory
(Master and Bachelor Thesis coordinated by Prof. Trausan-Matu)
 Genetic Algorith...
Chat sonification
(Stefan Trausan-Matu and Alexandru Calinescu)
60
Stefan Trausan-Matu
12/14/2013
Chat sonification
61
Stefan Trausan-Matu
12/14/2013
62
Stefan Trausan-Matu
12/14/2013
Chat sonification
(Orchestration by Serban Nichifor)
63
Stefan Trausan-Matu
12/14/2013
Conclusions
 The polyphonic model may apply to non-verbal
collaboration and intra-subjective (inner thinking)
as well as...
Thank you!
stefan.trausan@cs.pub.ro
http://www.racai.ro/~trausan
Questions?
65
Stefan Trausan-Matu
12/14/2013
of 65

Polyphonic model, analysis method, software and sonification of discourse

Published on: Mar 4, 2016
Published in: Technology      Education      
Source: www.slideshare.net


Transcripts - Polyphonic model, analysis method, software and sonification of discourse

  • 1. Polyphonic Analysis of Discourse in Texts and in Collaborative Learning Chats Ştefan Trăuşan-Matu University Politehnica of Bucharest Computer Science Department
  • 2. Contents  The Polyphonic Model  Polyphonic Analysis  Implementations of the Polyphonic Analysis  The PolyCAFe Analysis System  Other Applications  Conclusions 2 Stefan Trausan-Matu 12/14/2013
  • 3. Polyphony- An Unitary Model of Human Communication  Mediation:  Using natural language (words) in  texts  hypertexts  discussion forums  conversations but also  Non verbal communication (e.g. gestures) 3 Stefan Trausan-Matu 12/14/2013
  • 4. Polyphony- An Unitary Model of Human Group Communication  Considering rather a dialogistic, post- structuralist (Bakhtin, Kristeva) than a mechanistic perspective on communication  Rather a socio-cultural (Vygotsky) approach than a cognitivist one (like in Artificial Intelligence) but considering the both  Ethnomethodology (Garfinkel), Conversation Analysis (Sacks, Schegloff, Jefferson) 4 Stefan Trausan-Matu 12/14/2013
  • 5. Polyphony- An Unitary Model of Group Communication  Applicable to:  Small groups (e.g. virtual teams collaborating by chat or forums – the INTER-ANIMATION phenomenon appears)  Large groups – social networks  Global level - intertextuality 5 Stefan Trausan-Matu 12/14/2013
  • 6. Polyphony- An Unitary Model of Group Communication and Intertextuality  It may be used in IT implementations using  Natural language Processing  Machine learning  Social Network analysis  Specific techniques (inter-animation and collaboration analysis) 6 Stefan Trausan-Matu 12/14/2013
  • 7. The Polyphonic Model 7 Stefan Trausan-Matu 12/14/2013
  • 8. Polyphony  Appears in music (e.g. J.S.Bach) and in texts (Bakhtin)  The Polyphonic  Model (Trausan-Matu, Handbook of Hybrid Learning, 2010)  Analysis method (Trausan-Matu and Rebedea, 2010)  Computer support tools for the polyphonic analysis of F2F, online and offline conversations:  The “Polyphony” system (Trausan-Matu and all, 2007)  ASAP (Dascalu, Chioasca and Trausan-Matu, 2008)  PolyCAFe (Trausan-Matu, Rebedea and Dascalu, 2011; Rebedea, Dascalu, Trausan-Matu and all, 2010)  Collaboration regions detection (Banica, Trausan-Matu and Rebedea, 2011)  Detection of the Important moments (Chiru and Trausan-Matu, 2012) 8 Stefan Trausan-Matu 12/14/2013
  • 9. Polyphony  A group of participants that, each of them keeps their individuality, personality, creativity, but also collaborate to achieve a common goal, trying to solve dissonances  A merge of:  Unity vs. Difference  Melody (longitudinal) and Harmony (transversal) Cycles – centrifugal/centripetal forces  Inter-animation of voices – inter-animation Stefan Trausan-Matu patterns 12/14/2013  Dissonance – Consonance  9
  • 10. The Polyphonic Model  Polyphony = Model of collaboration and interaction (Trausan-Matu, Stahl and Zemel, 2005)  Human communication in knowledge construction and collaboration are processes in which words and other utterances are linked in parallel threads which interact similarly to voices in polyphonic music  Repetition and rhythm are essential 10 Stefan Trausan-Matu 12/14/2013
  • 11. 11 Stefan Trausan-Matu 12/14/2013
  • 12. Polyphony 12 Stefan Trausan-Matu 12/14/2013
  • 13. Dialogism and Polyphony (Bakhtin)  Mikhail Bakhtin: • Utterances (not sentences) should be the unit of analysis • “These are different voices singing variously on a single theme. This is indeed 'multivoicedness,' exposing the diversity of life and the great complexity of human experience. 'Everything in life is counterpoint, that is, opposition,' “ (Bakhtin, 1984) • “… Any true understanding is dialogic in nature” (VoloshinovBakhtin, 1973) • Speech genres • Polyphony Inter-animation of voices • Basis for the CSCL paradigm (Koschman, 1999) • Opposed to de Saussure ideas: • Real life dialog should be the focus, not written text • Words are not arbitrary 13 Stefan Trausan-Matu 12/14/2013
  • 14. Bakhtin’s Polyphony  Everything is a dialog (applying not only to speech and text)  Utterances  Voices  Inter-animations among voices 14 Stefan Trausan-Matu 12/14/2013
  • 15. Utterances  Utterances (not sentences, as in ‘classical’ linguistics) should be the unit of analysis (Bakhtin)  Utterances are acts  An utterances may be a:        Word Turn, a reply in a conversation, chat or forum Sentence Text Image (picture, diagramatic representation, etc.) Gesture (individual or group) Thought – inner utterances – inner speech  Utterances should be considered at different granularities  Utterances are linked in threads formed by:  Explicit links (VMT chat environment; forum’s replies) - uptakes (Suthers, 2010)  Implicit links, detected by Natural Language Processing techniques – contingencies, uptakes (Suthers, 2010)  Utterances may become voices 15 Stefan Trausan-Matu 12/14/2013
  • 16. Voices  Distinctive presences in a group, influencing the other voices  Generated by utterances (singular or repeated)  Correspond to:  participants (may also be inner voices)  groups of participants (e.g. collective or collaborative utterances)  chains or threads of words or concepts:      repeated words lexical chains co-references reasoning or argumentation rhetorical schemas  Each utterance may contain multiple voices  Voices continue and influence each other through explicit or implicit links. 16 Stefan Trausan-Matu 12/14/2013
  • 17. Inter-animation patterns  Longitudinal  Adjacency pairs  Repetitions  Elaboration  Convergence  Cumulative talk  Repair  Transversal, differential  Dissonance 17 Stefan Trausan-Matu 12/14/2013
  • 18. The Polyphonic Method applications  Chat conversations with multiple participants for:  CSCL: K-12 students solving mathematics problems both individually and collaboratively in the VMT project at Drexel University, Philadelphia, US  CS students at University Politehnica of Bucharest , Romania at o CHI course in Romanian and French – role playing and debate o Natural Language Processing - role playing and debate o Algorithm Design – problem solving   Fostering creativity – brainstorming, synectics  F2F collaborative learning (Suthers & all, 2011)  Analysis of Rhythm  Metacognition (conversation & essays)  OpenSimDeveloper dataset 18 Stefan Trausan-Matu  Intertextuality 12/14/2013
  • 19. Computer Support for the Polyphonic Analysis 19 Stefan Trausan-Matu 12/14/2013
  • 20. LTfLL - EU FP7 Project (2008-2011) and NSF Virtual Math Teams Project http://www.ltfll-project.org/ http://mathforum.org  Language Technologies for Lifelong Learning  Netherlands, France, United Kingdom, Germany, Ausria, Romania, Bulgaria  PolyCAFe system (Polyphony-based Collaboration Analysis and Feedback generation)  The system has been validated with students and tutors in  University of Manchester, UK  Politehnica University of Bucharest, Romania 20 Stefan Trausan-Matu 12/14/2013
  • 21. Chat-based CSCL  K-12 students solving mathematics problems both individually and collaboratively in the Virtual Math Teams (VMT) project at Drexel University, Philadelphia, US (Directed by Gerry Stahl)  Computer Science students at University Politehnica of Bucharest (UPB), Romania at  Human-Computer Interaction course in Romanian and French – role playing and debate  Natural Language Processing - role playing and debate  Algorithm Design – problem solving 21 Stefan Trausan-Matu 12/14/2013
  • 22. Theories for analysing multi-parties conversation  Discourse analysis (Tannen)  Conversation analysis (Sacks, Jefferson, Schegloff)  Accountable talk (Resnick)  Transactivity (Teasley, Berkowitz & Gibbs, Joshi & Rose)  Events/contingencies, coordinations/uptakes (Suthers)  Inter-animation (Bakhtin, Wegerif, Trausan-Matu)  Polyphony (Bakhtin, Trausan-Matu) 22 Stefan Trausan-Matu 12/14/2013
  • 23. Analysis methods  TF-IDF  Latent Semantic Analysis  Naïve Bayes  Social Network Analysis  WordNet (wordnet.princeton.edu)  Support Vector Machines  Collin’s perceptron  TagHelper environment 23 Stefan Trausan-Matu 12/14/2013
  • 24. Analyis methods  TF-IDF  Latent Semantic Analysis Almost all are based also on  Naïve Bayes a two interlocutors  Social Network Analysis model, in which  WordNet (wordnet.princeton.edu) one person speaks  Support Vector Machines at a time, resulting  Collin’s perceptron one discussion thread  TagHelper environment 24 Stefan Trausan-Matu 12/14/2013
  • 25. 25 Stefan Trausan-Matu 12/14/2013
  • 26. 26 Stefan Trausan-Matu 12/14/2013
  • 27. NLP pipe  spelling correction, stemmer, tokenizer, Named Entity Recognizer, POS tagger and parser, and NP-chunker. Stanford NLP software (http://nlp.stanford.edu/software)  Spellchecker : Jazzy http://www.ibm.com/developerworks/java/library/jjazzy/  Alternative NLP pipes are under development,  GATE (http://gate.ac.uk)  LingPipe (http://aliasi.com/lingpipe/). 27 Stefan Trausan-Matu 12/14/2013
  • 28. Social network analysis  Consider explicit and implicit referencing as arcs between participants, which are the nodes  A kind of page-rank algorithm – an utterance is important if it is referred by important utterances; The strength of a voice (of an utterance) depends on the strength of the utterances that refer to it  Determines if a person is central/peripheral 28 Stefan Trausan-Matu 12/14/2013
  • 29. Polyphony, Inter-animation and Collaboration analysis  Assign an importance value for each utterance considering several indicators of inter-animation (collaboration)  Detection of voices (chains) inter-animation patterns (Trausan-Matu) in the chat  Consider several criteria such as the presence in the chat of questions, agreement, disagreement  Presence of others’ voices  Social Networks metrics  Machine learning approach (genetic algorithms and neural networks) for tuning the 29 Stefan Trausan-Matu 12/14/2013
  • 30. Computational details (Trausan-Matu, Dascalu and Dessus, ITS 2012; Dascalu, Trausan-Matu and all, 2010, 2011)) 30 Stefan Trausan-Matu 12/14/2013
  • 31. Representations: Conversation graph  For each participant there is a separate horizontal line in the representation  Each utterance is placed in the line corresponding to the issuer of that utterance, according to the emission time, alligned from left to right  The explicit references among utterances are depicted using connecting lines distinctively colored  The implicit references (deduced by the system) are represented using other color that the explicit ones  An estimation of the strength of each 31 utterance (when available) is represented as a bar chart Stefan Trausan-Matu 12/14/2013
  • 32. Representations: Weaving of Voices  Voices in the conversation graph  Participants = horizontal lines  Threads of repeated words or phrases = differently colored threads 32 Stefan Trausan-Matu 12/14/2013
  • 33. 33 Stefan Trausan-Matu 12/14/2013
  • 34. 34 Stefan Trausan-Matu 12/14/2013
  • 35. 35 Stefan Trausan-Matu 12/14/2013
  • 36. 36 Stefan Trausan-Matu 12/14/2013
  • 37. 37 Stefan Trausan-Matu 12/14/2013 37
  • 38. 38 Stefan Trausan-Matu 12/14/2013 38
  • 39. 39 Stefan Trausan-Matu 12/14/2013 39
  • 40. 40 Stefan Trausan-Matu 12/14/2013 40
  • 41. 41 Stefan Trausan-Matu 12/14/2013 41
  • 42. Validation (Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2010; Rebedea, Dascalu, Trausan-Matu and all, ECTEL 2011) 42 Stefan Trausan-Matu 12/14/2013
  • 43. Other applications 43 Stefan Trausan-Matu 12/14/2013
  • 44. Analysis Dimensions (types of voices) in Face-to-Face Settings (Trausan-Matu, in Suthers & all, (eds.) 2013)  Spoken dialog  Body language  Individual  Collective  The visual dimension  Visual data on the blackboard  What others participants do  Others’ body language  Internal dialogue (at an intra-mental level)  Echoes 44 Stefan Trausan-Matu 12/14/2013
  • 45. Inner utterances, inner speech  “Mead (1934) called thought a <<conversation with the generalized other,>> implying that when we think individually we attempt to respondinternally and vicariously-to the imagined responses of others to our ideas and arguments.” (Resnick & all, 1993)  “There are no ontological differences between inner and outer speech” (Clark and Holquist, 1984). 45 Stefan Trausan-Matu 12/14/2013
  • 46. Detecting important moments (Chiru and Trausan-Matu, ITS 2012) 46 Stefan Trausan-Matu 12/14/2013
  • 47. Graphical representation of topic’s rhythmicity (Chiru, Cojocaru, Trausan-Matu and Rebedea, ISMIS 2011) 47 High rhythmicity for all topics – these were debated in parallel as it can be seen by the lack of flat lines near the left side of the Stefan Trausan-Matu representation. Low rhythmicity – flat lines on the left side of the graphic showing that the topic that they represent has not been debated in those parts of the chat. 12/14/2013
  • 48. Determination of collaborative regions (Banica, Trausan-Matu and Rebedea, CSCL 2011) 48 Stefan Trausan-Matu 12/14/2013
  • 49. Metacognition (Trausan-Matu, Dascalu and Dessus, ITS 2012)  Combining Chats and Texts  Intertextuality 49 Stefan Trausan-Matu 12/14/2013
  • 50. ReaderBench (Dascalu, Trausan-Matu, Dessus, 2012) 50 Stefan Trausan-Matu 12/14/2013
  • 51. Creativity fostering (e.g. brainstorming)(Trausan-Matu, 2011) 51 Stefan Trausan-Matu 12/14/2013
  • 52. Opinion mining (Musat, Velcin, Rizoiu, and Trausan-Matu, 2011) 52 Stefan Trausan-Matu 12/14/2013
  • 53. Topic Modeling (Musat and Trausan-Matu, 2011)  No generally accepted definition for a “topic”  Document clusters  Abstractions based on document clusters  Labels;  Centroids, etc  (Word, Probability) pairs  Bayesian statistical models  Topics – distributions over words  Documents – distributions over topics  Generative model  Topic Intertwining  Conceptually similar to the ideas of Mikhail Bakhtin  Topics and voices 53 Stefan Trausan-Matu 12/14/2013
  • 54. Topic Modeling (Musat and Trausan-Matu, 2011)  LDA/pLSA/hLDA/CTM  Each newer version corrects some flaws of the earlier ones  However the traditional means of testing the accuracy have been proven wrong  Even more reason to look into the problem of evaluating the models  LDA  Readily available  Mallet  Easily reproducible experiments  Well known topic model; 54 Stefan Trausan-Matu 12/14/2013
  • 55. Intertextuality analysis (Ghiban & Trausan-Matu, 2012) Voice I Voice I Voice II In dialog Voice III Voice II Voice III Text 1 Text 2 Text 3 Text 1 Text 2 Text 3 In dialog in text 4 Text 4 55 Stefan Trausan-Matu 12/14/2013
  • 56. Intertextuality analysis (Ghiban & Trausan-Matu, 2012) 56 Theme 2 and Theme 3 may have the Stefan Trausan-Matu same words but only Section 1 and 6 are dialogical or polyphonical. They may present a 12/14/2013 higher force of expresivity.
  • 57. Analysis of interethnic discourse (Trausan-Matu, 2012)  Needed a corpus of texts with time stamps  Extract recurrent concepts in texts  Identify historical events  Generate time series  Analysis of correlations between time series  Analysis of the polyphonic structure 57 Stefan Trausan-Matu 12/14/2013
  • 58. Time Series Analysis of News (Badea & Trausan-Matu, 2013) 58 Stefan Trausan-Matu 12/14/2013
  • 59. Music Composition at K-Teams Laboratory (Master and Bachelor Thesis coordinated by Prof. Trausan-Matu)  Genetic Algorithms  Celular automata  Artificial chemistry  Constraint-based systems  Accompaniments generation with Markov Models  Random generation  Automatic counterpoint generation according to Fux rules  Chat sonification 59 Stefan Trausan-Matu 12/14/2013
  • 60. Chat sonification (Stefan Trausan-Matu and Alexandru Calinescu) 60 Stefan Trausan-Matu 12/14/2013
  • 61. Chat sonification 61 Stefan Trausan-Matu 12/14/2013
  • 62. 62 Stefan Trausan-Matu 12/14/2013
  • 63. Chat sonification (Orchestration by Serban Nichifor) 63 Stefan Trausan-Matu 12/14/2013
  • 64. Conclusions  The polyphonic model may apply to non-verbal collaboration and intra-subjective (inner thinking) as well as inter-subjective levels  A combination of Conversational Analysis with Natural Language Processing is possible (cognitive and socio-cultural)  Learning analytics tools that combine the two perspectives and the Polyphonic Model may be developed 64 Stefan Trausan-Matu 12/14/2013
  • 65. Thank you! stefan.trausan@cs.pub.ro http://www.racai.ro/~trausan Questions? 65 Stefan Trausan-Matu 12/14/2013