網(wǎng)上有很多關(guān)于pos機(jī)匯總異常,Spark Graphx 圖計算 圖構(gòu)造器匯總的知識，也有很多人為大家解答關(guān)于pos機(jī)匯總異常的問題，今天pos機(jī)之家(www.afbey.com)為大家整理了關(guān)于這方面的知識，讓我們一起來看下吧!
本文目錄一覽：
pos機(jī)匯總異常
spark grahx構(gòu)造圖的方式匯總?cè)缦拢?/p>1.最原始的方法構(gòu)造graph
使用Graph的伴生對象的apply方法。允許從頂點(diǎn)和邊的RDD上創(chuàng)建一個圖。
import org.apache.log4j.{Level, Logger}import org.apache.spark.graphx.{Graph, Edge}import org.apache.spark.{SparkContext, SparkConf}/*** Created by lichangyue on 2016/9/13.*/object FirstGraph1 {def main(args: Array[String]) { Logger.getLogger("org.apache.spark").setLevel(Level.ERROR); Logger.getLogger("org.eclipse.jetty.server").setLevel(Level.ERROR);val conf = new SparkConf().setAppName("FirstGrahp").setMaster("local")val sc = new SparkContext(conf)val users = sc.parallelize(Array((3L,("rxin","student")),(7L,("jgonzal","postdoc")), (5L,("franklin","prof")),(2L,("istoica","prof"))))val relationships = sc.parallelize(Array(Edge(3L,7L,"collab"),Edge(5L,3L,"advisor"),Edge(2L,5L,"collegaue"),Edge(5L,7L,"pi")))val defaultUser = ("John Doe","Missing")
//創(chuàng)建圖val graph =Graph(users,relationships,defaultUser)//模式匹配 println( "count :" +graph.vertices.filter{ case(id,(name,pos)) => pos =="postdoc"}.count)graph.edges.filter(e => e.srcId > e.dstId ).count() }}2.從文件中讀取讀取數(shù)據(jù)構(gòu)建graph
從文件中讀取讀取數(shù)據(jù)構(gòu)建graph的接口，GraphLoader中的edgeListFile方法。
GraphLoader.edgeListFile 從文件中加載一張圖，文件內(nèi)容如下：
2 1
4 1
1 2
會自動根據(jù)邊創(chuàng)建頂點(diǎn)，所有的邊和頂點(diǎn)的屬性默認(rèn)為1。
object GraphFromFile2 {def main(args: Array[String]) { Logger.getLogger("org.apache.spark").setLevel(Level.ERROR); Logger.getLogger("org.eclipse.jetty.server").setLevel(Level.ERROR);// val conf = new SparkConf().setAppName("graph2").setMaster("spark://10.58.22.219:7077")val conf = new SparkConf().setAppName("graph2").setMaster("local[4]")val sc = new SparkContext(conf)// val graph = GraphLoader.edgeListFile(sc,"hdfs://S7SA053:8020/stat/web-Google.txt") //指定分區(qū)數(shù)量val graph = GraphLoader.edgeListFile(sc,"hdfs://S7SA053:8020/stat/web-Google.txt",numEdgepartitions=4)//查看頂點(diǎn)數(shù)量val vcount = graph.vertices.count()println("vcount:" +vcount)//查看邊的數(shù)量val ecount = graph.edges.count()println("ecount:" + ecount) }}3. 使用邊創(chuàng)建圖
Graph.fromEdges(ClassTag[VD],ClassTag[ED]):Graph[VD,ED]) 允許僅僅從一個邊RDD上創(chuàng)建一個圖，它自動地創(chuàng)建邊提及的頂點(diǎn)，并分配這些頂點(diǎn)默認(rèn)的值。
def fromEdges[VD: ClassTag, ED: ClassTag]( edges: RDD[Edge[ED]], defaultValue: VD, edgeStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY, vertexStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY): Graph[VD, ED]
val edge=List(//邊的信息(111,122),(111,133),(122,133),(133,144),(133,155),(133,116), (144,155),(155,116),(177,188),(177,199),(188,199))//構(gòu)建邊的rddval edgeRdd = sc.parallelize(edge).map(x =>{Edge(x._1.toLong , x._2.toLong, None)})//通過邊構(gòu)建圖 ， 0是頂點(diǎn)默認(rèn)屬性val graph = Graph.fromEdges(edgeRdd,0)4.從源點(diǎn)和目的點(diǎn)的元組構(gòu)建
Graph.fromEdgeTuples],VD,Option[PartitionStrategy])(ClassTag[VD]):Graph[VD,Int]) 允許僅僅從一個邊元組組成的RDD上創(chuàng)建一個圖。分配給邊的值為1。它自動地創(chuàng)建邊提及的頂點(diǎn)，并分配這些頂點(diǎn)默認(rèn)的值。
def fromEdgeTuples[VD: ClassTag]( rawEdges: RDD[(VertexId, VertexId)], defaultValue: VD, uniqueEdges: Option[PartitionStrategy] = None, edgeStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY, vertexStorageLevel: StorageLevel = StorageLevel.MEMORY_ONLY): Graph[VD, Int]5.生成隨機(jī)圖
GraphGenerators.logNormalGraph隨機(jī)圖生成方法源碼：默認(rèn)出度為4，標(biāo)準(zhǔn)偏差為1.3，并行生成numVertices，partition默認(rèn)為sc的默認(rèn)partition。
Logger.getLogger("org.apache.spark").setLevel(Level.ERROR);Logger.getLogger("org.eclipse.jetty.server").setLevel(Level.ERROR);val conf = new SparkConf()val sc = new SparkContext("local","text",conf)val graph = GraphGenerators.logNormalGraph(sc,numVertices = 10) //生成10個頂點(diǎn)的圖 .mapVertices((id,_) => id.toDouble)graph.vertices.collect.foreach(println(_))graph.edges.collect.foreach(println(_))
以上就是關(guān)于pos機(jī)匯總異常,Spark Graphx 圖計算 圖構(gòu)造器匯總的知識，后面我們會繼續(xù)為大家整理關(guān)于pos機(jī)匯總異常的知識，希望能夠幫助到大家！
